<?xml version="1.0" standalone="yes"?>
<Paper uid="W97-0213">
  <Title>A Perspective on Word Sense Disambiguation Methods and Their Evaluation</Title>
  <Section position="6" start_page="84" end_page="84" type="concl">
    <SectionTitle>
4 Conclusions
</SectionTitle>
    <Paragraph position="0"> The most important of our observations about the state of the art in word sense disambiguation is that it is still a hard, open problem, for which the field has not yet narrowed much. We have made several suggestions that we believe will help assess progress and advance the state of the art. In summary: * We proposed that the accepted standard for WSD evaluation include a cross-entropy like measure that tests the accuracy of the probabilities assigned to sense tags and offers a mechanism for assigning partial credit.</Paragraph>
    <Paragraph position="1"> * We suggested a paradigm for common evaluation that combines the benefits of traditional &amp;quot;interesting word&amp;quot; evaluations with an emphasis on broad cov~age and scalability.</Paragraph>
    <Paragraph position="2"> * We outlined a criterion that should help in determinirlg a suitable sense inventory to use for comparison of algorithms, compatible with both hierarchical sense partitions and multilingually motivated sense distinctions.</Paragraph>
  </Section>
class="xml-element"></Paper>