<?xml version="1.0" standalone="yes"?>
<Paper uid="W02-0716">
  <Title>Towards a Speech-to-Speech Machine Translation Quality Metric</Title>
  <Section position="4" start_page="0" end_page="0" type="concl">
    <SectionTitle>
3 Summary and Conclusions
</SectionTitle>
    <Paragraph position="0"> The recommended U2U metric characteristics are summarized in Table 1.</Paragraph>
    <Paragraph position="1">  Seven (+/- two) primary categories Two secondary categories (major/minor) for each primary category.</Paragraph>
    <Paragraph position="2"> Numeric demerits for each major and minor category.</Paragraph>
    <Paragraph position="3"> Primary categories sorted by demerits. Meta-Rule 1 removes ambiguity of primary category assignment.</Paragraph>
    <Paragraph position="4"> Meta-Rule 2 removes ambiguity of secondary category assignment.</Paragraph>
    <Paragraph position="5"> While I have discussed several characteristics of what I consider to be required elements of any adequate U2U translation quality metric for production use, these constraints still permit infinite variation in the definition of any particular metric. Primary categories may be drawn from any number of classifications that divide the error space, e.g. lexical, syntactic, semantic, phonetic, etc. Numeric demerits may be taken from any desired range, be it 0-1 or 1-1000.</Paragraph>
    <Paragraph position="6"> Finally, let me say that just because a U2U metric conforms to the characteristics discussed in this paper, that metric does not automatically become a good metric. As previously discussed, the category definitions are of extreme importance, as are the examples used to illustrate the definitions and the training materials created for evaluators. Without clear, unambiguous and precise error definitions no metric will be of any practical value.</Paragraph>
  </Section>
class="xml-element"></Paper>