<?xml version="1.0" standalone="yes"?>
<Paper uid="W02-1006">
  <Title>An Empirical Evaluation of Knowledge Sources and Learning Algorithms for Word Sense Disambiguation</Title>
  <Section position="8" start_page="0" end_page="0" type="concl">
    <SectionTitle>
7 Discussions
</SectionTitle>
    <Paragraph position="0"> Based on our experimental results, there appears to be no single, universally best knowledge source. Instead, knowledge sources and learning algorithms interact and influence each other. For example, local collocations contribute the most for SVM, while parts-of-speech (POS) contribute the most for NB.</Paragraph>
    <Paragraph position="1"> NB even outperforms SVM if only POS is used. In addition, different learning algorithms benefit differently from feature selection. SVM performs best without feature selection, whereas NB performs best with some feature selection (a44 a12a104a45a74a46 ). We will investigate the effect of more elaborate feature selection schemes on the performance of different learning algorithms for WSD in future work.</Paragraph>
    <Paragraph position="2"> Also, using the combination of four knowledge sources gives better performance than using any single individual knowledge source for most algorithms. On the SENSEVAL-2 test set, SVM achieves 65.4% (all 4 knowledge sources), 64.8% (remove syntactic relations), 61.8% (further remove POS), and 60.5% (only collocations) as knowledge sources are removed one at a time.</Paragraph>
    <Paragraph position="3"> Before concluding, we note that the SENSEVAL-2 participating system UMD-SST (Cabezas et al., 2001) also used SVM, with surrounding words and local collocations as features. However, they reported recall of only 56.8%. In contrast, our implementation of SVM using the two knowledge sources of surrounding words and local collocations achieves recall of 61.8%. Following the description in (Cabezas et al., 2001), our own re-implementation of UMD-SST gives a recall of 58.6%, close to their reported figure of 56.8%. The performance drop from 61.8% may be due to the different collocations used in the two systems.</Paragraph>
  </Section>
class="xml-element"></Paper>