<?xml version="1.0" standalone="yes"?>
<Paper uid="H91-1059">
  <Title>THIRD MESSAGE UNDERSTANDING EVALUATION AND CONFERENCE (MUC-3): PHASE 1 STATUS REPORT</Title>
  <Section position="3" start_page="0" end_page="0" type="intro">
    <SectionTitle>
BACKGROUND
</SectionTitle>
    <Paragraph position="0"> The Naval Ocean Systems Center (NOSC) is extending the scope of previous efforts in the area of evaluating English text analysis systems. These evaluations are intended to advance our understanding of the merits of current text analysis techniques, as applied to the performance of a realistic information extraction task. The current one is also intended to provide insight into information retrieval technology (document retrieval and categorization) used instead of or in concert with language understanding technology. The inputs to the analysis/extraction process consist of naturally-occurring texts that were obtained by NOSC in the form of electronic messages. The outputs of the process are a set of templates or semantic frames resembling the contents of a partially formatted database.</Paragraph>
    <Paragraph position="1"> The premise on which the evaluations are based is that task-oriented tests enable straightforward comparisons among systems and provide useful quantitative data on the state of the art in text understanding. Even though the tests are designed to treat the systems under evaluation as black boxes, they are also designed to point up system performance on individual aspects of the task as well as on the task overall. Furthermore, these quantitative data can be interpreted in light of information known about each system's text analysis techniques in order to yield qualitative insights into the relative validity of those techniques as applied to the general problem of information extraction.</Paragraph>
  </Section>
class="xml-element"></Paper>