<?xml version="1.0" standalone="yes"?>
<Paper uid="A94-1009">
  <Title>Does Baum-Welch Re-estimation :Help Taggers?</Title>
  <Section position="3" start_page="54" end_page="54" type="intro">
    <SectionTitle>
2 The tagger and corpora
</SectionTitle>
    <Paragraph position="0"> The experiments were conducted using two taggers, one written in C at Cambridge University Computer Laboratory, and the other in C-t-+ at Sharp Laboratories. Both taggers implement the FB, Viterbi and BW algorithms. For training from a hand-tagged corpus, the model is estimated by counting the number of transitions from each tag i to each tag j, the total occurrence of each tag i, and the total occurrence of word w with tag i. Writing these as f(i,j), f(i) and f(i,w) respectively, the transition probability from tag i to tag j is estimated as f(i,j)/f(i) and the lexical probability as f(i, w)/f(i). Other estimation formulae have been used in the past. For example, CLAWS (Garside ct al., 1987) normalises the lexical probabilities by the total frequency of the word rather than of the tag. Consulting the Baum-Welch re-estimation formulae suggests that the approach described is more appropriate, and this is confirmed by slightly greater tagging accuracy. Any transitions not seen in the training corpus are given a small, non-zero probability The lexicon lists, for each word, all of tags seen in the training corpus with their probabilities. For words not found in the lexicon, all open-class tags are hypothesised, with equal probabilities. These words are added to the lexicon at the end of first iteration when re-estimation is being used, so that the probabilities of their hypotheses subsequently diverge from being uniform.</Paragraph>
    <Paragraph position="1"> To measure the accuracy of the tagger, we compare the chosen tag with one provided by a human annotator. Various methods of quoting accuracy have been used in the literature, the most common being the proport ion of words (tokens) receiving the correct tag. A better measure is the proportion of ambiguous words which are given the correct tag, where by ambiguous we mean that more than one tag was hypothesised. The former figure looks more impressive, but the latter gives a better measure of how well the tagger is doing, since it factors out the trivial assignment of tags to non-ambiguous words.</Paragraph>
    <Paragraph position="2"> For a corpus in which a fraction a of the words are ambiguous, and p is the accuracy on ambiguous words, the overall accuracy can be recovered from 1 - a + pa. All of the accuracy figures quoted below are for ambiguous words only.</Paragraph>
    <Paragraph position="3"> The training and test corpora were drawn from the LOB corpus and the Penn treebank. The hand tagging of these corpora is quite different. For example, the LOB tagset used 134 tags, while the Penn treebank tagset has 48. The general pattern of the results presented does not vary greatly with the corpus and tagset used.</Paragraph>
  </Section>
class="xml-element"></Paper>