<?xml version="1.0" standalone="yes"?>
<Paper uid="M93-1005">
  <Title>DOMAIN AND LANGUAGE EVALUATION RESULTS</Title>
  <Section position="2" start_page="0" end_page="0" type="intro">
    <SectionTitle>
INTRODUCTION
</SectionTitle>
    <Paragraph position="0"> The Fifth Message Understanding Conference (MUC-5) focused on the task of data extraction for tw o distinctly different applications, one within the domain of joint ventures (JV) and the other within the domain o f microelectronics (ME) . For each application, the task could be performed in either English and/or Japanese, giving four combinations : English Joint Ventures, Japanese Joint Ventures, FngJish Microelectronics, and Japanese Microelectronics .</Paragraph>
    <Paragraph position="1"> Interpreting the evaluation results across domains and within a single domain between languages is affecte d by a number of factors. Differences in task focus, complexity, and domain technicality make it impossible to appl y inferential statistics between domains . In addition, even though the task and the template design were the same across languages within a single domain, differences in the types of text sources for each language and accompanyin g variations in template fills and fill rules by language also make it impossible to apply inferential statistics between the language pairs . Moreover, there is considerable variation in the participants' level of effort and funding, and not all o f the participants worked in multiple languages and/or multiple domains .</Paragraph>
    <Paragraph position="2"> In light of these factors, I will present descriptive statistics comparing error per response fill to address th e following questions: (1) For both languages, what is the performance difference between domains? (2) Betwee n domains, what are performance differences for the single shared object and for unattempted slots? (3) For both domains, what is the performance difference between languages? (4) For a single domain, what are representativ e differences at object and slot levels between English and Japanese? The discussion of domain and language difference s will center upon general factors that influence performance in information extraction : the information defined for extraction, the information available in a corpus for extraction, and the way in which information is presented withi n a text.</Paragraph>
  </Section>
class="xml-element"></Paper>