<?xml version="1.0" standalone="yes"?>
<Paper uid="W02-0814">
  <Title>Evaluating the results of a memory-based word-expert approach to unrestricted word sense disambiguation.</Title>
  <Section position="7" start_page="1991" end_page="1991" type="concl">
    <SectionTitle>
5 Conclusion
</SectionTitle>
    <Paragraph position="0"> In this paper, we evaluated the results of the Antwerp automatic disambiguation system in the context of the SENSEVAL-2 English all words task. Our approach was to create word-experts per word-POS pair. These word-experts consist of different classifiers/voters, which all take different information sources as input. We concluded that there was no information source which was optimal for all wordnouns verbs adverbs adjectives U Total  experts. But we also showed that selecting the optimal classifier/voter for each single word-expert led to major accuracy improvements.</Paragraph>
    <Paragraph position="1"> Since not all words were equally hard/easy to predict, we also evaluated the results of our WSD system in terms of the available number of training items, the number of senses and the sense distributions in the data set. Suprisingly, we observed that the available number of training items was not an accurate measure for task difficulty. But we furthermore concluded that the fluctuations in accuracy largely depend on the polysemy and entropy of the ambiguous words. On the basis of these results, we conclude that a more coarse-grained granularity of the distinction between word senses would increase performance of the WSD systems and make them a possible candidate for integration in practical applications such as machine translation systems.</Paragraph>
    <Paragraph position="2"> When evaluating our system on the test set, accuracy dropped by nearly 20% compared to scores on the train set, which could be largely explained by lack of training material for many senses. So the creation of more annotated data is necesssary and will certainly cause major improvements of current WSD systems and NLP systems in general (see also (Banko and Brill, 2001)).</Paragraph>
  </Section>
class="xml-element"></Paper>