<?xml version="1.0" standalone="yes"?>
<Paper uid="C96-1071">
  <Title>Evaluation of an Algorithm for the Recognition and Classification of Proper Names</Title>
  <Section position="4" start_page="421" end_page="421" type="evalu">
    <SectionTitle>
4 Results and Evaluation
</SectionTitle>
    <Paragraph position="0"> After these processing stages, the results generator produces a version of the original text in which all the proper names which have been detected are marked up with pre-defined SGML tags, specifying their classes. These marked up texts are then automatically scored against manually marked up texts.</Paragraph>
    <Paragraph position="1"> A series of evaluations has been done on tile system using a blind test set consisting of 30 Wall Street Journal texts. In these texts there are 449 organisation names, 373 person names, and 110 location names and 111 time expressions in total.</Paragraph>
    <Paragraph position="2"> The overall precision and recall scores for the four classes of proper naines are shown in Table 1.</Paragraph>
    <Section position="1" start_page="421" end_page="421" type="sub_section">
      <SectionTitle>
4.1 System module contribution
</SectionTitle>
      <Paragraph position="0"> We have analysed tile results in terms of how much each module of the system contributes to the proper nmne task.</Paragraph>
      <Paragraph position="1"> Table 2 illustrates the contribution of each system module to the task for all classes of proper names. In addition to recall and t)recision scores, we have added Van Rijsbergen's F-measure which combines these scores into a single measure (Rijsbergen, 1979). The F-measure (also called P&amp;R) allows the differential weighting of precision and recall. With precision and recall weighted equally it is computed by the formula:  results using tagging, exact, phrase matching, trigger word detection, and parsing (setting 2). Note that this amounts to making use of only internal evidence. However, to achieve higher recall, we need coreference resolution for proper names (setting 3) and other context information (setting 4).</Paragraph>
    </Section>
    <Section position="2" start_page="421" end_page="421" type="sub_section">
      <SectionTitle>
4.2 Different classes of proper names
</SectionTitle>
      <Paragraph position="0"> We have also examined how the contribution of each component varies from one class of proper nanm to another.</Paragraph>
      <Paragraph position="1"> For organisation names, using the same settings as above, scores are shown in Table 3.</Paragraph>
      <Paragraph position="2">  For person names, location names and time expressions the results are shown in Tables 4-6.  Figure 2 shows graphically how the system components contribute for each of the four different classes of proper names as well as for all classes combined.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>