<?xml version="1.0" standalone="yes"?>
<Paper uid="C96-2213">
  <Title>Using a Hybrid System of Corpusand Knowledge-Based Techniques to Automate the Induction of a Lexical Sublanguage Grammar</Title>
  <Section position="4" start_page="1165" end_page="1165" type="evalu">
    <SectionTitle>
5. Evaluation
</SectionTitle>
    <Paragraph position="0"> Manning evaluates his system by computing precision and recall scores with the OALD dictionary as golden standard. However, precision is no__!t a good yardstick for evaluating the performance of the induction process, because it measures the outcome against a &amp;quot;flawed&amp;quot; lexicon; the induced features, because of the data-driven nature of the process, are more &amp;quot;precise&amp;quot; when measured against the &amp;quot;real world&amp;quot; of the sublanguage domain than the hand-built entries that are the product mostly of introspection and anecdotal evidence. The system described in this paper was tested instead by comparing the number of successful parses of a held-out test corpus before and after customizing the lexicon. Out-of-the-box PUNDIT returned 42 parses for the 170 sentences in the training corpus (some of which were false positives), versus 94 successful parses using the attuned lexicon. It should be pointed out that these 94 sentences contain an average of 2.14 verbs.</Paragraph>
  </Section>
class="xml-element"></Paper>