<?xml version="1.0" standalone="yes"?>
<Paper uid="W99-0621">
  <Title>A Learning Approach to Shallow Parsing*</Title>
  <Section position="7" start_page="174" end_page="175" type="evalu">
    <SectionTitle>
5 Experimental Results
</SectionTitle>
    <Paragraph position="0"/>
    <Section position="1" start_page="174" end_page="174" type="sub_section">
      <SectionTitle>
5.1 Inside/Outside
</SectionTitle>
      <Paragraph position="0"> The results of each of the predictors used in the Inside/0utside method are presented in Table 3. The results are comparable to other results reported using the Inside/Outside method (Ramshaw and Marcus, 1995) (see Table 7. We have observed that most of the mistaken predictions of base NPs involve predictions with respect to conjunctions, gerunds, adverbial NPs and some punctuation marks. As reported in (Argamon et al., 1998), most base NPs present in ~he data are less or equal than 4 words long. This implies that our predictors tend to break up long base NPs into smaller ones.</Paragraph>
      <Paragraph position="1"> The results also show that lexical information improves the performance by nearly 2%. This is similar to results in the literature (Ramshaw and Marcus, 1995). What we found surprising is that the second predictor, that uses additional information about the OIB status of the local context, did not do much better than the first predictor, which relies only on POS and lexical information. A control experiment has verified that this is not due to the noisy features that the first predictor supplies to the second predictor. Finally, the Inside/Outside method was also tested on predicting SV phrases, yielding poor results that are not shown here. An attempt at explaining this phenomena by breaking down performance according to the length of the phrases is discussed in Sec. 5.3.</Paragraph>
    </Section>
    <Section position="2" start_page="174" end_page="175" type="sub_section">
      <SectionTitle>
5.2 Open/Close
</SectionTitle>
      <Paragraph position="0"> The results of the Open/Close method for NP and SV phrases are presented in Table 4. In addition to the good overall performance, the results show significant improvement by incorporating the lexical information into the features. In addition to the recall/precision results we have also presented the accuracy of each of the Open and Close predictors. These are important since they determine the overall accuracy in phrase detection. It is evident that the predictors perform very well, and that the overall performance degrades due to inconsistent pairings. null An important question in the learning approach presented here is investigating the gain achieved due to chaining. That is, whether the features extracted from open brackets can improve the performance of the the close bracket predictor. To this effect, we measured the accuracy of the close bracket predictor itself, on a word basis, by supplying it features generated from correct open brackets. We compared this with the same experiment, only this time without incorporating the features from open brackets to the close bracket predictor. The results, shown in Table 5 indicate a significant contribution due to chaining the features. Notice that the overall accuracy for the close bracket predictor is very high. This is due to the fact that, as shown in Table 2, there are many more negative examples than positive examples. Thus, a  versus using additional features created from the open bracket candidate. Overall performance and performance on positive examples only is shown.</Paragraph>
      <Paragraph position="1"> predictor that always predicts &amp;quot;no&amp;quot; would have an accuracy of 93.4%. Therefore, we considered also the accuracy over positive examples, which indicates the significant role of the chaining.</Paragraph>
    </Section>
    <Section position="3" start_page="175" end_page="175" type="sub_section">
      <SectionTitle>
5.3 Discussion
</SectionTitle>
      <Paragraph position="0"> Both methods we study here - Inside/Outside and Open/Close - have been evaluated before (using different learning methods) on similar tasks. However, in this work we have allowed for a fair comparison between two different models by using the same basic learning method and the same features.</Paragraph>
      <Paragraph position="1"> Our main conclusion is with respect to the robustness of the methods to sequences of different lengths. While both methods give good results for the base NP problem, they differ significantly on the SV tasks. Furthermore, our investigation revealed that the Inside/Outside method is very sensitive to the length of the phrases. Table 6 shows a breakdown of the performance of the two methods on SV phrases of different lengths. Perhaps this was not observed earlier since (Ramshaw and Marcus, 1995) studied only base NPs, most of which are short.</Paragraph>
      <Paragraph position="2"> The conclusion is therefore that the Open/Close method is more robust, especially when the target sequences are longer than a few tokens.</Paragraph>
      <Paragraph position="3"> Finally, Tables 7 and 8 present a comparison of our methods to some of the best NP and SV results published on these tasks.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>