<?xml version="1.0" standalone="yes"?>
<Paper uid="P04-1027">
  <Title>An Empirical Study of Information Synthesis Tasks</Title>
  <Section position="1" start_page="0" end_page="0" type="abstr">
    <SectionTitle>
Abstract
</SectionTitle>
    <Paragraph position="0"> This paper describes an empirical study of the &amp;quot;Information Synthesis&amp;quot; task, defined as the process of (given a complex information need) extracting, organizing and inter-relating the pieces of information contained in a set of relevant documents, in order to obtain a comprehensive, non redundant report that satisfies the information need.</Paragraph>
    <Paragraph position="1"> Two main results are presented: a) the creation of an Information Synthesis testbed with 72 reports manually generated by nine subjects for eight complex topics with 100 relevant documents each; and b) an empirical comparison of similarity metrics between reports, under the hypothesis that the best metric is the one that best distinguishes between manual and automatically generated reports. A metric based on key concepts overlap gives better results than metrics based on n-gram overlap (such as ROUGE) or sentence overlap.</Paragraph>
  </Section>
class="xml-element"></Paper>