<?xml version="1.0" standalone="yes"?>
<Paper uid="A00-2024">
  <Title>Cut and Paste Based Text Summarization</Title>
  <Section position="4" start_page="179" end_page="179" type="metho">
    <SectionTitle>
3 System architecture
</SectionTitle>
    <Paragraph position="0"> The architecture of our cut and paste based text summarization system is shown in Figure 1. Input to the system is a single document from any domain.</Paragraph>
    <Paragraph position="1"> In the first stage, extraction, key sentences in the article are identified, as in most current summarizers.</Paragraph>
    <Paragraph position="2"> In the second stage, cut and paste based generation, a sentence reduction module and a sentence combination module implement the operations we observed in human-written abstracts.</Paragraph>
    <Paragraph position="3"> The cut and paste based component receives as input not only the extracted key sentences, but also the original article. This component can be ported to other single-document summarizers to serve as the generation component, since most current summarizers extract key sentences - exactly what the extraction module in our system does.</Paragraph>
    <Paragraph position="4"> Other resources and tools in the summarization system include a corpus of articles and their human-written abstracts, the automatic decomposition program, a syntactic parser, a co-reference resolution system, the WordNet lexical database, and a large-scale lexicon we combined from multiple resources. The components in dotted lines are existing tools or resources; all the others were developed by ourselves.</Paragraph>
  </Section>
  <Section position="5" start_page="179" end_page="182" type="metho">
    <SectionTitle>
4 Major components
</SectionTitle>
    <Paragraph position="0"> The main focus of our work is on decomposition of summaries, sentence reduction, and sentence combination. We also describe the sentence extraction module, although it is not the main focus of our work.</Paragraph>
    <Section position="1" start_page="179" end_page="180" type="sub_section">
      <SectionTitle>
4.1 Decomposition of human-written
</SectionTitle>
      <Paragraph position="0"> summary sentences The decomposition program, see (Jing and McKeown, 1999) for details, is used to analyze the construction of sentences in human-written abstracts. The results from decomposition are used to build the training and testing corpora for sentence reduction and sentence combination.</Paragraph>
      <Paragraph position="1"> The decomposition program answers three questions about a sentence in a human-written abstract: (1) Is the sentence constructed by cutting and pasting phrases from the input article? (2) If so, what phrases in the sentence come from the original article? (3) Where in the article do these phrases come from? We used a Hidden Markov Model (Baum, 1972) solution to the decomposition problem. We first mathematically formulated the problem, reducing it to a problem of finding, for each word in a summary  Summary sentence: (F0:S1 arthur b sackler vice president for law and public policy of time warner inc ) (FI:S-1 and) (F2:S0 a member of the direct marketing association told ) (F3:$2 the communications subcommittee of the senate commerce committee ) (F4:S-1 that legislation ) (F5:Slto protect ) (F6:$4 children' s ) (F7:$4 privacy ) (F8:$4 online ) (F9:S0 could destroy the spontaneous nature that makes the internet unique ) Source document sentences: Sentence 0: a proposed new law that would require web publishers to obtain parental consent before collecting personal information from children (F9 could destroy the spontaneous nature that makes the internet unique ) (F2 a member of the direct marketing association told) a senate panel thursday Sentence 1:(F0 arthur b sackler vice president for law and public policy of time warner inc ) said the association supported efforts (F5 to protect ) children online but he urged lawmakers to find some middle ground that also allows for interactivity on the internet Sentence 2: for example a child's e-mail address is necessary in order to respond to inquiries such as updates on mark mcguire's and sammy sosa's home run figures this year or updates of an online magazine sackler said in testimony to (F3 the communications subcommittee of the senate  is uniquely identified by the position of the sentence where the word appears, and the position of the word within the sentence. Based on the observation of cut and paste practice by humans, we produced a set of general heuristic rules. Sample heuristic rules include: two adjacent words in a summary sentence are most likely to come from two adjacent words in the original document; adjacent words in a summary sentence are not very likely to come from sentences that are far apart in the original document. We use these heuristic rules to create a Hidden Markov Model. The Viterbi algorithm (Viterbi, 1967) is used to efficiently find the most likely document position for each word in the summary sentence.</Paragraph>
      <Paragraph position="2"> Figure 2 shows sample output of the program.</Paragraph>
      <Paragraph position="3"> For the given summary sentence, the program correctly identified that the sentence was combined from four sentences in the input article. It also divided the summary sentence into phrases and pinpointed the exact document origin of each phrase.</Paragraph>
      <Paragraph position="4"> A phrase in the summary sentence is annotated as (FNUM:SNUM actual-text), where FNUM is the sequential number of the phrase and SNUM is the number of the document sentence where the phrase comes from. SNUM = -1 means that the component does not come from the original document. The phrases in the document sentences are annotated as (FNUM actual-text).</Paragraph>
    </Section>
    <Section position="2" start_page="180" end_page="181" type="sub_section">
      <SectionTitle>
4.2 Sentence reduction
</SectionTitle>
      <Paragraph position="0"> The task of the sentence reduction module, described in detail in (Jing, 2000), is to remove extraneous phrases from extracted sentences. The goal of reduction is to &amp;quot;reduce without major loss&amp;quot;; that is, we want to remove as many extraneous phrases as possible from an extracted sentence so that it can be concise, but without detracting from the main idea that the sentence conveys. Ideally, we want to remove a phrase from an extracted sentence only if it is irrelavant to the main topic.</Paragraph>
      <Paragraph position="1"> Our reduction module makes decisions based on  multiple sources of knowledge: (1) Grammar checking. In this step, we mark which components of a sentence or a phrase are obligatory to keep it grammatically correct. To do  this, we traverse the sentence parse tree, produced by the English Slot Grammar(ESG) parser developed at IBM (McCord, 1990), in top-down order and mark for each node in the parse tree, which of its children are obligatory. The main source of knowledge the system relies on in this step is a large-scale, reusable lexicon we combined from multiple resources (Jing and McKeown, 1998). The lexicon contains subcategorizations for over 5,000 verbs. This information is used to mark the obligatory arguments of verb phrases.</Paragraph>
      <Paragraph position="2"> (2) Context information. We use an extracted sentence's local context in the article to decide which components in the sentence are likely to be most relevant to the main topic. We link the words in the extracted sentence with words in its local context, if they are repetitions, morphologically related, or linked with each other in WordNet through certain type of lexical relation, such as synonymy, antonymy, or meronymy. Each word in the extracted sentence gets an importance score, based on the number of links it has with other words and the types of links. Each phrase in the sentence is then assigned a score  Original sentence : When it arrives sometime next year in new TV sets, the V-chip will give parents a new and potentially revolutionary device to block out programs they don't want their children to see.</Paragraph>
      <Paragraph position="3"> Reduction program: The V-chip will give parents a new and potentially revolutionary device to block out programs they don't want their children to see.  by adding up the scores of its children nodes in the parse tree. This score indicates how important the phrase is to the main topic in discussion.</Paragraph>
      <Paragraph position="4">  (3) Corpus evidence. The program uses a cor- null pus of input articles and their corresponding reduced forms in human-written abstracts to learn which components of a sentence or a phrase can be removed and how likely they are to be removed by professionals. This corpus was created using the decomposition program. We compute three types of probabilities from this corpus: the probability that a phrase is removed; the probability that a phrase is reduced (i.e., the phrase is not removed as a whole, but some components in the phrase are removed); and the probability that a phrase is unchanged at all (i.e., neither removed nor reduced). These corpus probabilities help us capture human practice. (4) Final decision. The final reduction decision is based on the results from all the earlier steps. A phrase is removed only if it is not grammatically obligatory, not the focus of the local context (indicated by a low context importance score), and has a reasonable probability of being removed by humans.</Paragraph>
      <Paragraph position="5"> The phrases we remove from an extracted sentence include clauses, prepositional phrases, gerunds, and to-infinitives.</Paragraph>
      <Paragraph position="6"> The result of sentence reduction is a shortened version of an extracted sentence 2. This shortened text can be used directly as a summary, or it can be fed to the sentence combination module to be merged with other sentences.</Paragraph>
      <Paragraph position="7"> Figure 3 shows two examples produced by the reduction program. The corresponding sentences in human-written abstracts are also provided for comparison. null 2It is actually also possible that the reduction program decides no phrase in a sentence should be removed, thus the result of reduction is the same as the input.</Paragraph>
      <Paragraph position="8"> sentence reduction program</Paragraph>
    </Section>
    <Section position="3" start_page="181" end_page="181" type="sub_section">
      <SectionTitle>
4.3 Sentence combination
</SectionTitle>
      <Paragraph position="0"> To build the combination module, we first manually analyzed a corpus of combination examples produced by human professionals, automatically created by the decomposition program, and identified a list of combination operations. Table 1 shows the combination operations.</Paragraph>
      <Paragraph position="1"> To implement a combination operation, we need to do two things: decide when to use which combination operation, and implement the combining actions. To decide when to use which operation, we analyzed examples by humans and manually wrote a set of rules. Two simple rules are shown in Figure 4. Sample outputs using these two simple rules are shown in Figure 5. We are currently exploring using machine learning techniques to learn the combination rules from our corpus.</Paragraph>
      <Paragraph position="2"> The implementation of the combining actions involves joining two parse trees, substituting a subtree with another, or adding additional nodes. We implemented these actions using a formalism based on</Paragraph>
    </Section>
    <Section position="4" start_page="181" end_page="182" type="sub_section">
      <SectionTitle>
Tree Adjoining Grammar (Joshi, 1987).
4.4 Extraction Module
</SectionTitle>
      <Paragraph position="0"> The extraction module is the front end of the summarization system and its role is to extract key sentences. Our method is primarily based on lexical relations. First, we link words in a sentence with other words in the article through repetitions, morphological relations, or one of the lexical relations encoded in WordNet, similar to step 2 in sentence reduction.</Paragraph>
      <Paragraph position="1"> An importance score is computed for each word in a sentence based on the number of lexical links it has with other words, the type of links, and the directions of the links.</Paragraph>
      <Paragraph position="2"> After assigning a score to each word in a sentence, we then compute a score for a sentence by adding up the scores for each word. This score is then normal-</Paragraph>
    </Section>
  </Section>
  <Section position="6" start_page="182" end_page="182" type="metho">
    <SectionTitle>
Categories Combination Operations
</SectionTitle>
    <Paragraph position="0"> Add descriptions or names for people or organizations</Paragraph>
    <Section position="1" start_page="182" end_page="182" type="sub_section">
      <SectionTitle>
Aggregations
Substitute incoherent phrases
</SectionTitle>
      <Paragraph position="0"> Substitute phrases with more general or specific information add description (see Figure 5) add name extract common subjects or objects (see Figure 5) change one sentence to a clause add connectives (e.g., and or while) add punctuations (e.g., &amp;quot;;&amp;quot;) substitute dangling anaphora substitute dangling noun phrases substitute adverbs (e.g., here) remove connectives substitute with more general information substitute with more specific information  IF: ((a person or an organization is mentioned the first time) and (the full name or the full description of the person or the organization exists somewhere in the original article but is missing in the summary)) THEN&amp;quot; replace the phrase with the full name plus the full description</Paragraph>
    </Section>
    <Section position="2" start_page="182" end_page="182" type="sub_section">
      <SectionTitle>
Rule 2:
</SectionTitle>
      <Paragraph position="0"> IF: ((two sentences are close to each other in the original article) and (their subjects refer to the same entity) and (at least one of the sentences is the reduced form resulting from sentence reduction)) null THEN: merge the two sentences by removing the subject in the second sentence, and then combining it with the first sentence using connective &amp;quot;and&amp;quot;.</Paragraph>
      <Paragraph position="1">  ized over the number of words a sentence contains.</Paragraph>
      <Paragraph position="2"> The sentences with high scores are considered important. null The extraction system selects sentences based on the importance computed as above, as well as other indicators, including sentence positions, cue phrases, and tf*idf scores.</Paragraph>
    </Section>
  </Section>
  <Section position="7" start_page="182" end_page="183" type="metho">
    <SectionTitle>
5 Evaluation
</SectionTitle>
    <Paragraph position="0"> Our evaluation includes separate evaluations of each module and the final evaluations of the overall system. null We evaluated the decomposition program by two experiments, described in (Jing and McKeown, 1999). In the first experiment, we selected 50 human-written abstracts, consisting of 305 sentences in total. A human subject then read the decomposition results of these sentences to judge whether they are correct. 93.8% of the sentences were correctly decomposed. In the second experiment, we tested the system in a summary alignment task. We ran the decomposition program to identify the source document sentences that were used to construct the sentences in human-written abstracts. Human subjects were also asked to select the document sentences that are semantlc-equivalent to the sentences in the abstracts. We compared the set of sentences identified by the program with the set of sentences selected by the majority of human subjects, which is used as the gold standard in the computation of precision and recall. The program achieved an average 81.5% precision, 78.5% recall, and 79.1% f-measure for 10 documents. The average performance of 14 human judges is 88.8% precision, 84.4% recall, and 85.7% f-measure. Recently, we have also tested the system on legal documents (the headnotes used by Westlaw company), and the program works well on those documents too.</Paragraph>
    <Paragraph position="1"> The evaluation of sentence reduction (see (Jing, 2000) for details) used a corpus of 500 sentences and their reduced forms in human-written abstracts. 400 sentences were used to compute corpus probabilities and 100 sentences were used for testing. The results show that 81.3% of the reduction decisions made by the system agreed with those of humans.</Paragraph>
    <Paragraph position="2"> The humans reduced the length of the 500 sentences by 44.2% on average, and the system reduced the length of the 100 test sentences by 32.7%.</Paragraph>
    <Paragraph position="3"> The evaluation of sentence combination module is not as straightforward as that of decomposition or reduction since combination happens later in the pipeline and it depends on the output from prior  Example 1: add descriptions or names for people or organization Original document sentences: &amp;quot;We're trying to prove that there are big benefits to the patients by involving them more deeply in their treatment&amp;quot;, said Paul Clayton, Chairman of the Department dealing with computerized medical information at Columbia.</Paragraph>
    <Paragraph position="4"> &amp;quot;The economic payoff from breaking into health care records is a lot less than for banks&amp;quot;, said Clayton at Columbia.</Paragraph>
    <Paragraph position="5"> Combined sentence: &amp;quot;The economic payoff from breaking into health care records is a lot less than for banks&amp;quot;, said Paul Clayton, Chairman of the Department dealing with computerized medical information at Columbia.  The new measure is an echo of the original bad idea, blurred just enough to cloud prospects both for enforcement and for court review.</Paragraph>
    <Paragraph position="6"> Unlike the 1996 act, this one applies only to commercial Web sites - thus sidestepping 1996 objections to the burden such regulations would pose for museums, libraries and freewheeling conversation deemed &amp;quot;indecent&amp;quot; by somebody somewhere. The new version also replaces the vague &amp;quot;indecency&amp;quot; standard, to which the court objected, with the better-defined one of material ruled &amp;quot;harmful to minors.&amp;quot; Combined sentences: The new measure is an echo of the original bad idea.</Paragraph>
    <Paragraph position="7"> The new version applies only to commercial web sites and replaces the vague &amp;quot;indecency&amp;quot; standard with the better-defined one of material ruled &amp;quot;harmful to minors.&amp;quot; Professional: While the new law replaces the &amp;quot;indecency&amp;quot; standard with &amp;quot;harmful to minors&amp;quot; and now only applies to commercial Web sites, the &amp;quot;new measure is an echo of the original bad idea.&amp;quot;  modules. To evaluate just the combination component, we assume that the system makes the same reduction decision as humans and the co-reference system has a perfect performance. This involves manual tagging of some examples to prepare for the evaluation; this preparation is in progress. The evaluation of sentence combination will focus on the accessment of combination rules.</Paragraph>
    <Paragraph position="8"> The overM1 system evMuation includes both intrinsic and extrinsic evaluation. In the intrinsic evMuation, we asked human subjects to compare the quality of extraction-based summaries and their revised versions produced by our sentence reduction and combination modules. We selected 20 documents; three different automatic summarizers were used to generate a summary for each document, producing 60 summaries in total. These summaries are all extraction-based. We then ran our sentence reduction and sentence combination system to revise the summaries, producing a revised version for each summary. We presented human subjects with the full documents, the extraction-based summaries, and their revised versions, and asked them to compare the extraction-based summaries and their revised versions. The human subjects were asked to score the conciseness of the summaries (extraction-based or revised) based on a scale from 0 to 10 the higher the score, the more concise a summary is.</Paragraph>
    <Paragraph position="9"> They were also asked to score the coherence of the summaries based on a scale from 0 to 10. On average, the extraction-based summaries have a score of 4.2 for conciseness, while the revised summaries have a score of 7.9 (an improvement of 88%). The average improvement for the three systems are 78%, 105%, and 88% respectively. The revised summaries are on average 41% shorter than the original extraction-based summaries. For summary coherence, the average score for the extraction-based summaries is 3.9, while the average score for the revised summaries is</Paragraph>
    <Section position="1" start_page="183" end_page="183" type="sub_section">
      <SectionTitle>
6.1 (an improvement of 56%). The average improve-
</SectionTitle>
      <Paragraph position="0"> ment for the three systems are 69%, 57%, and 53% respectively.</Paragraph>
      <Paragraph position="1"> We are preparing a task-based evaluation, in which we will use the data from the Summarization EvMuation Conference (Mani et al., 1998) and compare how our revised summaries can influence humans' performance in tasks like text categorization and ad-hoc retrieval.</Paragraph>
    </Section>
  </Section>
  <Section position="8" start_page="183" end_page="184" type="metho">
    <SectionTitle>
6 Related work
</SectionTitle>
    <Paragraph position="0"> (Mani et al., 1999) addressed the problem of revising summaries to improve their quality. They suggested three types of operations: elimination, aggregation, and smoothing. The goal of the elimination operation is similar to that of the sentence reduction op- null eration in our system. The difference is that while elimination always removes parentheticals, sentence-initial PPs and certain adverbial phrases for every extracted sentence, our sentence reduction module aims to make reduction decisions according to each case and removes a sentence component only if it considers it appropriate to do so. The goal of the aggregation operation and the smoothing operation is similar to that of the sentence combination operation in our system. However, the combination operations and combination rules that we derived from corpus analysis are significantly different from those used in the above system, which mostly came from operations in traditional natural language generation. null</Paragraph>
  </Section>
class="xml-element"></Paper>