<?xml version="1.0" standalone="yes"?>
<Paper uid="C00-2089">
  <Title>Tagging and Chunking with Bigrams</Title>
  <Section position="4" start_page="616" end_page="618" type="relat">
    <SectionTitle>
3 Experimental Work
</SectionTitle>
    <Paragraph position="0"> In this section we will describe a set of experiments that we carried out in order to demonstrate the capabilities of the proposed approach for tagging and shallow parsing. The experiments were carried out  on the WSJ corpus, using the POS tag set; defined in (Marcus etlal. , 1993), considering only the NP chunt{s (lefine~l by (Church, 1988) and using tile models that we have presented above. Nevertheless, the use of this apt)roach on other corpora (changing the reference language), other lexical tag sets or other kinds of chunks can be done in a direct way.</Paragraph>
    <Section position="1" start_page="617" end_page="617" type="sub_section">
      <SectionTitle>
3.1 Corpus Description.
</SectionTitle>
      <Paragraph position="0"> We used a t)ortion of the WSJ corpus (900,000 words), which was tagged according to the Penn Treebank tag set and bracketed with NP markers, to train and test the system.</Paragraph>
      <Paragraph position="1"> The tag set contained 45 different tags. About 36.5% of the words in the cortms were mnbiguous, with an ambiguity ratio of 2.44 tag/word over the ambiguous words, 1.52 overall.</Paragraph>
    </Section>
    <Section position="2" start_page="617" end_page="618" type="sub_section">
      <SectionTitle>
3.2 Experimental Results.
</SectionTitle>
      <Paragraph position="0"> In order to train the models and to test the system, we randomly divided the corpora into two parts: approximately 800,000 words for training aud 100,000 words tbr testing.</Paragraph>
      <Paragraph position="1"> Both the bigram models for representing contextual information mid syntactic description of the NP chunk and the lexical probabilities were estimated from training sets of different sizes. Due to the fact that we did not use a morphological analyser for English, we constructed a tag dictionary with the lexicon of the training set and the test set used. This dictionary gave us tile possible lexical tags for each word fl'om the corpus. In no case, was the test used to estimate the lexical probabilities.</Paragraph>
      <Paragraph position="2">  In Figure 4, we show the results of tagging on the test set in terms of the training set size using three at)proaches: the simplest (LEX) is a tagging process which does not take contextual information into account, so the lexical tag associated to a word will  ferents taggers (training set of 800,000 words).</Paragraph>
      <Paragraph position="3"> be that which has aI)peared more often in the training set. Tile second method corresponds to a tagger based on a bigram model (BIG). The third one uses the Integrated LM described in this pai)er (BIG-BIG). The tagging accuracy for BIG and BIG-BIG was close, 96.9% and 96.8% respectively, whereas without the use of the language model (LEX), tile tagging accuracy was 2.5 points lower. The trend in all the cases was that an increment in the size of the training set resulted in an increase in the tagging accuracy. After 300,000 training words, the result became stabilized.</Paragraph>
      <Paragraph position="4"> In Figure 5, we show the precision (#correct proposed NP/#proposed NP) and recall (#correct proposed NP/#NP in the reference) rates for NP chunking. The results obtained using the Integrated LM were very satisfactory achieving a precision rate of 94.6% and a recall rate of 93.6%. The performance of the NP chunker improves as the training set size increases. This is obviously due to the fact that tile model is better learnt when the size of the training set increases, and the tagging error decreases as we have seen above.</Paragraph>
      <Paragraph position="5"> The usual sequential 1)rocess for chunking a sentence can also be used. That is, first we tag the sentence and then we use the Integrated LM to carry out the chunking. In this case, only tim contextual t)robabilities are taken into account in the decoding  1)recess. In Table 2, we show the most relevant resuits that we obtained for tagging and tbr NP chunking. The first row shows the result when the tagging and the chunking are done in a integrated way. The following rows show the performmme of the sequential process using different taggers:  rate of 100%. To do this, we used the tagged sentences of the WSJ corlms directly.</Paragraph>
      <Paragraph position="6"> These results confirm that precision and recall rates increase when the accuracy of the tagger is beN;er. The pert'ormmme of 1;he, se(tuential process (u:dng the BIG tagger) is slightly 1letter than the pet'formance of the integrated process (BIG-BIG).</Paragraph>
      <Paragraph position="7"> We think that this is 1)robably b(;cause of the way we combined the I)robabilities of t;he ditthrent models. null</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>