<?xml version="1.0" standalone="yes"?>
<Paper uid="W00-1311">
  <Title>Detection of Language (Model) Errors</Title>
  <Section position="5" start_page="92" end_page="92" type="evalu">
    <SectionTitle>
4 Summary and Future Work
</SectionTitle>
    <Paragraph position="0"> We have evaluated both model-based and language-specific features for detecting language model errors. Individual model-based features did not yield good detection accuracy, suffering from the precision-recall trade-off. The language-specific features detect errors better. In particular, matched multi-character words are usually correct.</Paragraph>
    <Paragraph position="1"> If the model-based and language-specific features are aggregated as a single feature vector, the recall and precision of errors are 83% and 35%, respectively, which are the same if we just use language-specific features. Hence, instead of a single classifier, we separated 3 situations identified by the language-specific features and 3 classifiers are used to detect these errors individually. The Bayesian classifier (simpliest) achieved an overall 79% recall, 60% precision and 65% skip ratio and the MLP achieved an overall 75% recall, 80% precision and a 66% skip ratio. Similar recall and precision performances are achieved using decision trees, which are preferred since their skip ratio is higher (i.e. 76%). Although the precision (so far) is not high (60% 80%), it is not the most important result because (1) this only represents a minor waste of checking effort, compared with scanning the entire text, and (2) the identified errors will be checked further or corrected either manually or automatically.</Paragraph>
  </Section>
class="xml-element"></Paper>