<?xml version="1.0" standalone="yes"?>
<Paper uid="W98-1117">
  <Title>A Maximum-Entropy Partial Parser for Unrestricted Text</Title>
  <Section position="7" start_page="146" end_page="148" type="evalu">
    <SectionTitle>
6 Results
</SectionTitle>
    <Paragraph position="0"> In this section, we report the results of a cross-validation of the parser carried out on the NeGra Treebank (Skut et al., 1997). The corpus was converted into structural tags and partitioned into a training and a testing part (90% and 10%, respectively). We repeated this procedure ten times with different partitionings;the results of these test runs were averaged.</Paragraph>
    <Paragraph position="1"> The weights of the features used by the maximum entropy parser were determined with the help of the Maximum Entropy Modelling Toolkit, cf. (Ristad, 1996). The number of features reached 120,000 for the full training corpus (12,000 sentences). Interestingly, tagging accuracy decreased after after 4-5 iterations of Improved Iterative Scaling, so only 3 iterations were carried out in each of the test runs.</Paragraph>
    <Paragraph position="2"> The accuracy measures employed are explained as follows.</Paragraph>
    <Paragraph position="3"> tags: the percentage of structural tags with the correct value r~ of the REL attribute, bracketing: the percentage of correctly recognised nodes, labelled bracketing: like bracketing, but including the syntactic category of the nodes, structural match: the percentage of correctly recognised tree structures (top-level chunks only, labelling is ignored).</Paragraph>
    <Paragraph position="4">  application, such phrases are part of larger structures. The external boundaries (the first and the last word of the examples) are highlighted by an annotator, the parser recognises the internal boundaries and assigns labels.</Paragraph>
    <Section position="1" start_page="147" end_page="148" type="sub_section">
      <SectionTitle>
6.1 Treebank Application
</SectionTitle>
      <Paragraph position="0"> In the treebank application, information about the external boundaries of a phrase is supplied by an annotator. To imitate this situation, we extracted from the NeGra corpus all sequences of part-of-speech tags spanned by NPs PPs, APs and complex adverbials. Other tags were left out since they do not appear in chunks recognised by the parser. Thus, the sentence shown in figure 3 contributed three substrings to the chunk corpus: ART NN, APPR ADJA NN and APPR ADV CARD NN NN, which would also be typical annotator input. A designated separator character was used to mark chunk boundaries. null Table 2 shows the performance of the parser on the chunk corpus.</Paragraph>
    </Section>
    <Section position="2" start_page="148" end_page="148" type="sub_section">
      <SectionTitle>
6.2 Chunking Application
</SectionTitle>
      <Paragraph position="0"> Table 3 shows precision and recall for the chunking application, i.e., the recognition of kernel NPs and PPs in part-of-speech tagged text.</Paragraph>
      <Paragraph position="1"> Post-nominal PP attachment is ignored. Unlike in the treebank application, there is no pre-editing by a human expert. The absolute numbers differ from those in table 2 because certain structures are ignored. The total number of structural tags is higher since we now parse whole sentences rather then separate chunks.</Paragraph>
      <Paragraph position="2"> In addition to the four accuracy measures defined above, we also give the percentage of chunks with correctly recognised external boundaries (irrespective of whether or not there are errors concerning their internal structure).</Paragraph>
    </Section>
    <Section position="3" start_page="148" end_page="148" type="sub_section">
      <SectionTitle>
6.3 Comparison to a Standard Tagger
</SectionTitle>
      <Paragraph position="0"> In the following, we compare the performance of the maximum-entropy parser with the precision of a standard HMM-based approach trained on the same data, but using only the frequencies of complete trigrams, bigrams and unigrams, whose probabilities are smoothed by linear interpolation, as described in section 4.3.1.</Paragraph>
      <Paragraph position="1"> Figure 5 shows the percentage of correctly assigned values ri of the R.EL attribute depending on the size of the training corpus. Generally, the maximum entropy approach outperforms the linear extrapolation technique by about 0.5% 1.5%, which corresponds to a 1% - 3% difference in structural match. The difference decreases as the size of the training sample grows. For the full corpus consisting of 12,000 sentences, the linear interpolation tagger is still inferior to the maximum entropy one, but the difference in precision becomes insignificant (0.2%). Thus, the maximum entropy technique seems to particularly advantageous in the case of sparse data.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>