<?xml version="1.0" standalone="yes"?>
<Paper uid="X93-1018">
  <Title>COMPARING HUMAN AND MACHINE PERFORMANCE FOR NATURAL LANGUAGE INFORMATION EXTRACTION: Results from the Tipster Text Evaluation</Title>
  <Section position="18" start_page="191" end_page="191" type="concl">
    <SectionTitle>
CONCLUSIONS
</SectionTitle>
    <Paragraph position="0"> The present study has shown that on the English Microelectronics extraction task, the best machine system performs with an error rate,of about 62%, a little less than twice that of the 33% error produced by highly skilled and experienced human analysts.</Paragraph>
    <Paragraph position="1"> This level of performance suggests that machine extraction systems are still far away from achieving high-quality extraction with the more difficult texts and extraction problems characterized by the Tipster corpus. However, machine performance is close enough to the human level to suggest that practical extraction systems could be built today by careful selection of both the text and the extraction task, and perhaps making use of integrated human-machine systems that can harness the abilities of both humans and machines for extraction rather than depending upon a machine-only system.</Paragraph>
  </Section>
class="xml-element"></Paper>