<?xml version="1.0" standalone="yes"?>
<Paper uid="H92-1063">
  <Title>A NEW APPROACH TO TEXT UNDERSTANDING</Title>
  <Section position="4" start_page="317" end_page="318" type="intro">
    <SectionTitle>
3. EVALUATION
</SectionTitle>
    <Paragraph position="0"> The system as a whole was formally evaluated in the Government-sponsored Third Message Understanding Conference (MUC-3), and scored among the top systems in extracting data from text \[Proceedings of MUC-3, 1991\].</Paragraph>
    <Paragraph position="1"> In this paper we report on two additional experiments run since then to assess component conlributions to the system.</Paragraph>
    <Section position="1" start_page="317" end_page="318" type="sub_section">
      <SectionTitle>
3.1 Lexicon
</SectionTitle>
      <Paragraph position="0"> If the grammar rules and semantic rules are both compositional and domain-independent, one would expect the recall of the system (the percent of information correctly found by the system out of all desired information in the text) to grow linearly at first as the lexicon grows followed by tapering off to an asyptote. 4 To test this, we ran the system after randomly removing lexical entries (though not removing a word's part of speech). The results with various percentages of the lexicon and with linear curve fitting appear in Figure 4.</Paragraph>
      <Paragraph position="1"> The template generator has three tasks: finding and/or merging events hypothesized by discourse processing into a</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>