<?xml version="1.0" standalone="yes"?>
<Paper uid="W97-0710">
  <Title>Sentence extraction as a classification task</Title>
  <Section position="3" start_page="0" end_page="58" type="intro">
    <SectionTitle>
1 Introduction
</SectionTitle>
    <Paragraph position="0"> A useful first step m the automatic or semi-automatic generation of abstracts from source texts m the selection of a small number of 'meamngful' sentences from the source text To achieve tins, each sentence m the source text is scored according to some measure of importance, and the best-rated sentences are selected Thin results m collections of the N most 'meamngful' sentences, m the order m wlnch they appeared m the source text - we will call these excerpts An excerpt can be used to give readers an idea of what the longer text m about, or It can be used as input into a process to .produce a more coherent abstract It has been argued for almost 40 years that it m posmble to automatically create excerpts which meet bamc reformation compresmon needs (Luhn, 1958) Since then, different measurements for the importance of a sentence have been suggested, m particular stochastic measurements for the mgmficance of -key words or phrases (Lulm, 1058, Zechner, 1995) Other research, starting with (Edmundson, 1969), stressed the Importance of heuristics for the location of the candidate sentence m the source text (Baxendale, 1958) and for the occurrence of cue phrases (Palce and Jones, 1993, Johnson et al, 1993) Single heunstms tend to work well on documents that resemble each other m style and content For the more robust creation of excerpts, combinations of these heuristics can be used The eruclal question m how to combine the C/hfferent heuristics In the past, the relative usefulness of single methods had to be balanced manually Kupmc et al (1095) use supervised learnmg to automatically adjust feature w~ghts, using a corpus of research papers and corresponding summaries Humans have good intuition about what makes a sentence 'abstract-worthy', I e suitable for inclusion in a summary Abstract-worthiness m a lughlevel quality, comprising notions such as semantic content, relative importance and appropriateness for representing the contents of a document * .For the automatic evaluation of the quality of machine generated excerpts, one has to find an operational approximation to this subjective notion of abstractworthiness, 1 e a defuntlon of a desired result We will call the criteria of what constitutes success the gold standard, and the set of sentences that fulfill</Paragraph>
    <Paragraph position="2"> these criteria the gold standard sentences Apart from evaluation, a gold standard m also needed for supervmed learning In Kupiec et al (1995), a gold standard sentence is a sentence m the source text that zs matched ruth a summary sentence on the basra of semantic and syntactic snnflanty In thear corpus of 188 engineermg papers with summaries written by professional abstractors, 79% of sentences occurred m both summary and source text with at most minor moddica~ tzons However, our collection of papers, whose abstracts were written by the authors themselves, shows a szgnh~cant difference these abstracts have $1~nl~-. cantly fewer ahgnable sentences (31 7%) This does. not mean that there are fewer .abstract-worthy sentenees m the source text We used a simple (labourintensive) way of defimng thin alternative gold standard, vzz aslang a human judge to identify additional abstract-worthy sentences in the source text Our mare question was whether Kuplec et al's methodology could be used for our kind of gold standard sentences also, and if there was a fundamental chfference in extraction performance between sentences in both gold standards or between documents with higher or lower alignment We also conducted an experiment to see how additional training matehal would influence the statistical model The remainder of this paper is organized as follows in the next section, we s-mmanze Kuplec et al's method and results Then, we describe our data and dmcuss the results from three experiments with dflferent evaluation strategies and tralmng material Differences between our and Kuplec et al's data with respect to the ahgnablhty of document and summary sentences, and consequences thereof are conmdered m the discussion</Paragraph>
  </Section>
class="xml-element"></Paper>