<?xml version="1.0" standalone="yes"?>
<Paper uid="W00-0733">
  <Title>Text Chunking by System Combination</Title>
  <Section position="4" start_page="151" end_page="152" type="intro">
    <SectionTitle>
3 Results
</SectionTitle>
    <Paragraph position="0"> In order to find out which of the three processing methods and which of the nine combination methods performs best, we have applied them to the training data of the CoNLL-2000 shared task (Tjong Kim Sang and Buchholz, 2000) in a 10-fold cross-validation experiment (Weiss and Kulikowski, 1991). For the single-pass method, we trained IBI-IG classifiers to produce the most likely output tags for the five data representations. In the input of the classifiers a word was represented as itself, its part-of-speech tag and a context of four left and four right word/partof-speech tag pairs. For the four IO representations we used a second phase with a limited input context (3) but with additionally the two previous and the two next chunk tags predicted by the first phase. The classifier output was converted to the O representation (open brackets) and the C representation (close brackets) and the results were combined with the nine combination methods. In the double-pass method finding the most likely tag for each word was split in finding chunk boundaries and assigning types to the chunks. The n-pass method divided this process into eleven passes each of which recognized one chunk type.</Paragraph>
    <Paragraph position="1"> For each processing strategy, all combination results were better than those obtained with the five individual classifiers. The differences between combination results within each processing strategy were small and between the three strategies the best results were not far apart: the best FZ=i rates were 92.40 (single-pass), 92.35 (double-pass) and 92.75 (n-pass).</Paragraph>
    <Paragraph position="2"> Since the three processing methods reach a similar performances, we can choose any of them for our remaining experiments. The n-pass method performed best but it has the disadvantage of needing as many passes as there are chunk types. This will require a lot of computation. The single-pass method was second-best but in order to obtain good results with this method, we would need to use a stacked classifier because those performed better (F~=1=92.40) than the voting methods (Fz=1=91.98). This stacked classifier requires preprocessed combinator training data which can be obtained by processing the original train- null ing data with 10-fold cross-validation. Again this will require a lot of work for new data sets. We have chosen for the double-pass method because in this processing strategy it is possible to obtain good results with majority voting. The advantage of using majority voting is that it does not require extra preprocessed combinator training data so by using it we avoid the extra computation required for generating this data. We have applied the double-pass method with majority voting to the CoNLL-2000 test data while using the complete training data. The results can be found in table 1. The recognition method performs well for the most frequently occurring chunk types (NP, VP and PP) and worse for the other seven (the test data did not contain UCP chunks). The recognition rate for NP chunks (F~=1=93.23) is close to the result for a related standard baseNP data set obtained by Tjong Kim Sang (2000) (93.26).</Paragraph>
    <Paragraph position="3"> Our method outperforms the results mentioned in Buchholz et al. (1999) in four of the five cases (ADJP, NP, PP and VP); only for ADVP chunks it performs slightly worse. This is surprising given that Buchholz et al. (1999) used 956696 tokens of training data and we have used only 211727 (78% less).</Paragraph>
  </Section>
class="xml-element"></Paper>