<?xml version="1.0" standalone="yes"?>
<Paper uid="P93-1020">
  <Title>INTENTION-BASED SEGMENTATION: HUMAN RELIABILITY AND CORRELATION WITH LINGUISTIC CUES</Title>
  <Section position="15" start_page="153" end_page="154" type="concl">
    <SectionTitle>
CONCLUSION
</SectionTitle>
    <Paragraph position="0"> We have shown that human subjects can reliably perform linear discourse segmentation in a corpus of transcripts of spoken narratives, using an informal notion of speaker intention. We found that percent agreement with the segmentations produced by the majority of subjects ranged from 82%-92%, with an average across all narratives of 89% (~=.0006).</Paragraph>
    <Paragraph position="1"> We found that these agreement results were highly significant, with probabilities of randomly achieving our findings ranging from p = .114 x 10 -6 to p &lt; .6 x 10 -9.</Paragraph>
    <Paragraph position="2"> We have investigated the correlation of our intention-based discourse segmentations with referential noun phrases, cue words, and pauses. We developed segmentation algorithms based on the use of each of these linguistic cues, and quantitatively evaluated their performance in identifying the statistically validated boundaries independently produced by our subjects. We found that compared to human performance, the recall of the three algorithms SFallout and error rate do not vary much across T i.</Paragraph>
    <Paragraph position="3">  was comparable, the precision was much lower, and the fallout and error of only the noun phrase algorithm was comparable. We also found a tendency for recall to increase and precision to decrease with exact boundary strength, suggesting that the cognitive salience of boundaries is graded.</Paragraph>
    <Paragraph position="4"> While our initial results are promising, there is certainly room for improvement. In future work on our data, we will attempt to maximize the correlation of our segmentations with linguistic cues by improving the performance of our individual algorithms, and by investigating ways to combine our algorithms (cf. Grosz and Hirschberg (1992)). We will also explore the use of alternative evaluation metrics (e.g. string matching) to support close as well as exact correlation.</Paragraph>
  </Section>
class="xml-element"></Paper>