<?xml version="1.0" standalone="yes"?>
<Paper uid="C94-1013">
  <Title>Evahmtion Metrics t'oi- Knowledge-Based Machine Translation</Title>
  <Section position="6" start_page="97" end_page="98" type="evalu">
    <SectionTitle>
6 Discussion
</SectionTitle>
    <Paragraph position="0"> Our ongoing evalt, atitm of the lirst large-scale KANT application Ires benefitted from the detailed error analysis presented here. Following tile tabulation of error codes l)rOduced during catlsal comp(mcnt analysis, we can attril)ute the ntajority of the completeness problems to identiliable gaps in lexieal coverage, :rod the majority of the accuracy prol)lefns to areas of the domain ntodel which are known Io be incolnplctc or insufiiciently general. On the other hand, the grammars of both source and target language, as well as tile software modules, are relatively solid, as very few errors can be attributed thereto. As lexieal coverage and domain model generalization reach completion, the component and global ewlh,ation of the KANT system will t)ecome a more accurate rellection of the potential of the nnde,lying technology in large-scale apl) lications.</Paragraph>
    <Paragraph position="1"> As illustr,'tted in Figm-e 5, traditional transfer-based MT systems start with general coverage, and gradt, ally seek to improve accuracy and later fluency. In contrast, the KBMT philosophy has been to start with high accuracy and gradually improve coverage and Iluen~ay. ht tile KANT systema, we combine both approaches by starting with coverage of a large specific dontain :rod achieving high accuracy and Iluency  within that domain.</Paragraph>
    <Paragraph position="2"> The evaluation methodol{}gy devtloped here is ,no:mr t{} I)e ustd in conjunction with glnbal black-box evaluation methods, indtl}endtnt of the course of develol}ment. The coml}ontnt evaluations arc meant to provide insight for the sysltm devtlopers, avid to identify prol)ltmatic phenomena prior to system coml}letion an{l dtlivefy. In particular, the method l}resented here c'm combine coml}onent evalttation and !gl{}l}a\] evaluation to support efficient system testing and nlaintenance beyond development.</Paragraph>
  </Section>
class="xml-element"></Paper>