<?xml version="1.0" standalone="yes"?>
<Paper uid="I05-5008">
  <Title>Automatic generation of paraphrases to be used as translation references in objective evaluation measures of machine translation</Title>
  <Section position="9" start_page="63" end_page="63" type="concl">
    <SectionTitle>
8 Conclusion
</SectionTitle>
    <Paragraph position="0"> We reported a technique to generate paraphrases in the view of constituting reference sets for machine translation evaluation measures like BLEU and NIST. In an experiment with a linguistic resource of 97,769 sentences we generated 8,65 paraphrases in average for 16,153 seed sentences.</Paragraph>
    <Paragraph position="1"> The grammaticality was evaluated by sampling and was shown to be of at least 99% grammatically and semantically correct sentences (p-value = 2.22%), a quality comparable to that of the original linguistic resource. In addition, at least 96% of the candidates (p-value = 1.92%) were correct paraphrases either by meaning equivalence or entailment. null Finally, the lexical and syntactical variation within each paraphrase set was assessed using BLEU and NIST against the seed sentence. It was found that the lexical and syntactical variation did not depend upon the number of paraphrases generated, but on the length of the seed sentence.</Paragraph>
    <Paragraph position="2"> Going back to the view of constituting reference sets for machine translation evaluation, not only are the paraphrase sets produced by this method correct sentences and valid paraphrases, but they also exhibit an internal lexical and syntactical variation which was shown to be slightly superior to that of two evaluation campaign sets of paraphrases produced by hand.</Paragraph>
  </Section>
class="xml-element"></Paper>