<?xml version="1.0" standalone="yes"?>
<Paper uid="J02-4002">
  <Title>c(c) 2002 Association for Computational Linguistics Summarizing Scientific Articles: Experiments with Relevance and Rhetorical Status</Title>
  <Section position="2" start_page="0" end_page="412" type="abstr">
    <SectionTitle>
1. Introduction
</SectionTitle>
    <Paragraph position="0"> Summarization systems are often two-phased, consisting of a content selection step followed by a regeneration step. In the first step, text fragments (sentences or clauses) are assigned a score that reflects how important or contentful they are. The highest-ranking material can then be extracted and displayed verbatim as &amp;quot;extracts&amp;quot; (Luhn 1958; Edmundson 1969; Paice 1990; Kupiec, Pedersen, and Chen 1995). Extracts are often useful in an information retrieval environment since they give users an idea as to what the source document is about (Tombros and Sanderson 1998; Mani et al. 1999), but they are texts of relatively low quality. Because of this, it is generally accepted that some kind of postprocessing should be performed to improve the final result, by shortening, fusing, or otherwise revising the material (Grefenstette 1998; Mani, Gates, and Bloedorn 1999; Jing and McKeown 2000; Barzilay et al. 2000; Knight and Marcu 2000).</Paragraph>
    <Paragraph position="1"> The extent to which it is possible to do postprocessing is limited, however, by the fact that contentful material is extracted without information about the general discourse context in which the material occurred in the source text. For instance, a sentence describing the solution to a scientific problem might give the main contri- null [?] Simone Teufel, Computer Laboratory, Cambridge University, JJ Thomson Avenue, Cambridge, CB3 OFD, England. E-mail: Simone.Teufel@cl.cam.ac.uk + Marc Moens, Rhetorical Systems and University of Edinburgh, 2 Buccleuch Place, Edinburgh, EH8 9LS, Scotland. E-mail: marc@cogsci.ed.ac.uk  Computational Linguistics Volume 28, Number 4 bution of the paper, but it might also refer to a previous approach that the authors criticize. Depending on its rhetorical context, the same sentence should be treated very differently in a summary. We propose in this article a method for sentence and content selection from source texts that adds context in the form of information about the rhetorical role the extracted material plays in the source text. This added contextual information can then be used to make the end product more informative and more valuable than sentence extracts.</Paragraph>
    <Paragraph position="2"> Our application domain is the summarization of scientific articles. Summarization of such texts requires a different approach from, for example, that used in the summarization of news articles. For example, Barzilay, McKeown, and Elhadad (1999) introduce the concept of information fusion, which is based on the identification of recurrent descriptions of the same events in news articles. This approach works well because in the news domain, newsworthy events are frequently repeated over a short period of time. In scientific writing, however, similar &amp;quot;events&amp;quot; are rare: The main focus is on new scientific ideas, whose main characteristic is their uniqueness and difference from previous ideas.</Paragraph>
    <Paragraph position="3"> Other approaches to the summarization of news articles make use of the typical journalistic writing style, for example, the fact that the most newsworthy information comes first; as a result, the first few sentences of a news article are good candidates for a summary (Brandow, Mitze, and Rau 1995; Lin and Hovy 1997). The structure of scientific articles does not reflect relevance this explicitly. Instead, the introduction often starts with general statements about the importance of the topic and its history in the field; the actual contribution of the paper itself is often given much later. The length of scientific articles presents another problem. Let us assume that our overall summarization strategy is first to select relevant sentences or concepts, and then to synthesize summaries using this material. For a typical 10- to 20-sentence news wire story, a compression to 20% or 30% of the source provides a reasonable input set for the second step. The extracted sentences are still thematically connected, and concepts in the sentences are not taken completely out of context. In scientific articles, however, the compression rates have to be much higher: Shortening a 20-page journal article to a half-page summary requires a compression to 2.5% of the original. Here, the problematic fact that sentence selection is context insensitive does make a qualitative difference. If only one sentence per two pages is selected, all information about how the extracted sentences and their concepts relate to each other is lost; without additional information, it is difficult to use the selected sentences as input to the second stage.</Paragraph>
    <Paragraph position="4"> We present an approach to summarizing scientific articles that is based on the idea of restoring the discourse context of extracted material by adding the rhetorical status to each sentence in a document. The innovation of our approach is that it defines principles for content selection specifically for scientific articles and that it combines sentence extraction with robust discourse analysis. The output of our system is a list of extracted sentences along with their rhetorical status (e.g. sentence 11 describes the scientific goal of the paper, and sentence 9 criticizes previous work), as illustrated in Figure 1. (The example paper we use throughout the article is F. Pereira, N. Tishby, and L. Lee's &amp;quot;Distributional Clustering of English Words&amp;quot; [ACL-1993, cmp lg/9408011]; it was chosen because it is the paper most often cited within our collection.) Such lists serve two purposes: in themselves, they already provide a better characterization of scientific articles than sentence extracts do, and in the longer run, they will serve as better input material for further processing.</Paragraph>
    <Paragraph position="5"> An extrinsic evaluation (Teufel 2001) shows that the output of our system is already a useful document surrogate in its own right. But postprocessing could turn  Teufel and Moens Summarizing Scientific Articles AIM 10 Our research addresses some of the same questions and uses similar raw data, but we investigate how to factor word association tendencies into associations of words to certain hidden senses classes and associations between the classes themselves.</Paragraph>
    <Paragraph position="6"> 11 While it may be worthwhile to base such a model on preexisting sense classes (Resnik, 1992), in the work described here we look at how to derive the classes directly from distributional data.</Paragraph>
    <Paragraph position="7"> 162 We have demonstrated that a general divisive clustering procedure for probability distributions can be used to group words according to their participation in particular grammatical relations with other words.</Paragraph>
    <Paragraph position="8"> BASIS 19 The corpus used in our first experiment was derived from newswire text automatically parsed by Hindle's parser Fidditch (Hindle, 1993).</Paragraph>
    <Paragraph position="9"> 113 The analogy with statistical mechanics suggests a deterministic annealing procedure for clustering (Rose et al., 1990), in which the number of clusters is determined through a sequence of phase transitions by continuously increasing the parameter EQN following an annealing schedule.</Paragraph>
    <Paragraph position="10"> CONTRAST 9 His notion of similarity seems to agree with our intuitions in many cases, but it is not clear how it can be used directly to construct word classes and corresponding models of association.</Paragraph>
    <Paragraph position="11"> 14 Class construction is then combinatorially very demanding and depends on frequency counts for joint events involving particular words, a potentially unreliable source of information as we noted above.</Paragraph>
    <Paragraph position="12"> Figure 1 Extract of system output for example paper.</Paragraph>
    <Paragraph position="13"> 0 This paper's topic is to automatically classify words according to their contexts of use. 4 The problem is that for large enough corpora the number of possible joint events is much larger than the number of event occurrences in the corpus, so many events are seen rarely or never, making their frequency counts unreliable estimates of their probabilities. 162 This paper's specific goal is to group words according to their participation in particular grammatical relations with other words, 22 more specifically to classify nouns according to their distribution as direct objects of verbs.</Paragraph>
    <Paragraph position="14"> Figure 2 Nonexpert summary, general purpose.</Paragraph>
    <Paragraph position="15"> the rhetorical extracts into something even more valuable: The added rhetorical context allows for the creation of a new kind of summary. Consider, for instance, the user-oriented and task-tailored summaries shown in Figures 2 and 3. Their composition was guided by fixed building plans for different tasks and different user models, whereby the building blocks are defined as sentences of a specific rhetorical status. In our example, most textual material is extracted verbatim (additional material is underlined in Figures 2 and 3; the original sentences are given in Figure 5). The first example is a short abstract generated for a nonexpert user and for general information; its first two sentences give background information about the problem tackled. The second abstract is aimed at an expert; therefore, no background is given, and instead differences between this approach and similar ones are described.</Paragraph>
    <Paragraph position="16"> The actual construction of these summaries is a complex process involving tasks such as sentence planning, lexical choice and syntactic realization, tasks that are outside the scope of this article. The important point is that it is the knowledge about the rhetorical status of the sentences that enables the tailoring of the summaries according to users' expertise and task. The rhetorical status allows for other kinds of applications too: Several articles can be summarized together, contrasts or complementarity among  Computational Linguistics Volume 28, Number 4 44 This paper's goal is to organise a set of linguistic objects such as words according to the contexts in which they occur, for instance grammatical constructions or n-grams. 22 More specifically: the goal is to classify nouns according to their distribution as direct objects of verbs. 5 Unlike Hindle (1990), 9 this approach constructs word classes and corresponding models of association directly. 14 In comparison to Brown et al. (1992), the method is combinatorially less demanding and does not depend on frequency counts for joint events involving particular words, a potentially unreliable source of information. Figure 3 Expert summary, contrastive links.</Paragraph>
    <Paragraph position="17"> articles can be expressed, and summaries can be displayed together with citation links to help users navigate several related papers.</Paragraph>
    <Paragraph position="18"> The rest of this article is structured as follows: section 2 describes the theoretical and empirical aspects of document structure we model in this article. These aspects include rhetorical status and relatedness: * Rhetorical status in terms of problem solving: What is the goal and contribution of the paper? This type of information is often marked by metadiscourse and by conventional patterns of presentation (cf.</Paragraph>
    <Paragraph position="19"> section 2.1).</Paragraph>
    <Paragraph position="20"> * Rhetorical status in terms of intellectual attribution: What information is claimed to be new, and which statements describe other work? This type of information can be recognized by following the &amp;quot;agent structure&amp;quot; of text, that is, by looking at all grammatical subjects occurring in sequence (cf. section 2.2).</Paragraph>
    <Paragraph position="21"> * Relatedness among articles: What articles is this work similar to, and in what respect? This type of information can be found by examining fixed indicator phrases like in contrast to ..., section headers, and citations (cf.</Paragraph>
    <Paragraph position="22"> section 2.3).</Paragraph>
    <Paragraph position="23"> These aspects of rhetorical status are encoded in an annotation scheme that we present in section 2.4. Annotation of relevance is covered in section 2.5.</Paragraph>
    <Paragraph position="24"> In section 3, we report on the construction of a gold standard for rhetorical status and relevance and on the measurement of agreement among human annotators. We then describe in section 4 our system that simulates the human annotation. Section 5 presents an overview of the intrinsic evaluation we performed, and section 6 closes with a summary of the contribution of this work, its limitations, and suggestions for future work.</Paragraph>
  </Section>
class="xml-element"></Paper>