<?xml version="1.0" standalone="yes"?>
<Paper uid="N03-1020">
  <Title>Automatic Evaluation of Summaries Using N-gram Co-Occurrence Statistics</Title>
  <Section position="1" start_page="0" end_page="0" type="abstr">
    <SectionTitle>
Abstract
</SectionTitle>
    <Paragraph position="0"> Following the recent adoption by the machine translation community of automatic evaluation using the BLEU/NIST scoring process, we conduct an in-depth study of a similar idea for evaluating summaries. The results show that automatic evaluation using unigram co-occurrences between summary pairs correlates surprising well with human evaluations, based on various statistical metrics; while direct application of the BLEU evaluation procedure does not always give good results.</Paragraph>
  </Section>
class="xml-element"></Paper>