<?xml version="1.0" standalone="yes"?>
<Paper uid="W02-0808">
  <Title>Sense Discrimination with Parallel Corpora</Title>
  <Section position="4" start_page="7" end_page="8" type="relat">
    <SectionTitle>
3 Discussion and Further Work
</SectionTitle>
    <Paragraph position="0"> Our results show that sense distinctions based on translation variants from parallel corpora are similar to those obtained from human annotators, which suggests several potential applications.</Paragraph>
    <Paragraph position="1"> Because our approach is fully automated through all its steps, it could be used to automatically obtain large samples of &amp;quot;sense-differentiated&amp;quot; data without the high cost of human annotation.</Paragraph>
    <Paragraph position="2"> Although our method does not choose sense assignments from a pre-defined list, most language processing applications (e.g. information retrieval) do not require this knowledge; they need only the information that different occurrences of a given word are used in the same or a different sense.</Paragraph>
    <Paragraph position="3"> A by-product of applying our method is that once words in a text in one language are tagged using this method, different senses of the corresponding translations in the parallel texts are also identified, potentially providing a source of information for use in other language processing tasks and for building resources in the parallel languages (e.g., WordNets for the Eastern European languages in our study). In addition, if different senses of target words are identified in parallel texts, contextual information for different senses of a word can be gathered for use in disambiguating other, unrelated texts. The greatest obstacle to application of this approach is, obviously, the lack of parallel corpora: existing freely available parallel corpora including several languages are typically small (e.g., the Orwell), domain dependent (e.g. the MULTEXT Journal of the Commission (JOC) corpus; Ide and Veronis, 1994), and/or represent highly stylized language (e.g. the Bible; Resnik et al., 1999).</Paragraph>
    <Paragraph position="4"> Appropriate parallel data including Asian languages is virtually non-existent. Given that our method applies only to words for which different senses are lexicalized differently in at least one other language, its broad application depends on the future availability of large-scale parallel corpora including a variety of language types.</Paragraph>
    <Paragraph position="5"> Many studies have pointed out that coarser-grained sense distinctions can be assigned more reliably by human annotators than finer distinctions such as those in WordNet. In our study, the granularity of the sense distinctions was largely ignored, except insofar as we attempted to cut off the number of clusters produced by the algorithm at a value similar to the number identified by the annotators.</Paragraph>
    <Paragraph position="6"> The sense distinctions derived from the clustering algorithm are hierarchical, often identifying four or five levels of refinement, whereas the WordNet sense distinctions are organized as a flat list with no indication of their degree of relatedness. Our attempt to flatten the cluster data in fact loses much information about the relatedness of senses.</Paragraph>
    <Paragraph position="7">  As a result, both annotators and the clustering algorithm are penalized as much for failing to distinguish fine-grained as coarse-grained distinctions. We are currently exploring two possible sources of information about sense relatedness: the output of the clustering algorithm itself, and WordNet hypernyms, which may not only improve but also broaden the applicability of our method.</Paragraph>
    <Paragraph position="8">  Interestingly, the clustering for &amp;quot;glass&amp;quot; in Figure 1 reveals additional sub-groupings that are not distinguished in WordNet: the top sub-group of the top cluster includes occurrences that deal with some physical aspect of the material (&amp;quot;texture of&amp;quot;, &amp;quot;surface of&amp;quot;, &amp;quot;rainwatery&amp;quot;, &amp;quot;soft&amp;quot;, etc.). In the lower cluster, the two main sub-groups distinguish a (drinking) glass as a manipulatable object (by washing, holding, on a shelf, etc.) from its sense as a vessel (mainly used as the object of &amp;quot;pour into&amp;quot;, &amp;quot;fill&amp;quot;, &amp;quot;take/pick up&amp;quot;, etc. or modified by &amp;quot;empty&amp;quot;, &amp;quot;of gin&amp;quot;, etc.).</Paragraph>
    <Paragraph position="9"> We note in our data that although it is not statistically significant, there is some correlation (.51) between the number of WordNet senses for a word and overall agreement levels. The lowest overall agreement levels were for &amp;quot;line&amp;quot; (29 senses), &amp;quot;step&amp;quot; (10), position (15), &amp;quot;place&amp;quot; (17), and &amp;quot;corner&amp;quot; (11). Perfect agreement was achieved for several words with under 5 senses, e.g., &amp;quot;hair&amp;quot; (5), &amp;quot;morning&amp;quot; (4), &amp;quot;sister&amp;quot; (4), &amp;quot;tree&amp;quot; (2), and &amp;quot;waist&amp;quot; (2)--all of which were judged by both the annotators and the algorithm to occur in only one sense in the text. On the other hand, agreement levels for some words with under five WordNet senses had low agreement: e.g., &amp;quot;rubbish&amp;quot; (2), &amp;quot;rhyme&amp;quot; (2), &amp;quot;destruction&amp;quot; (3), and &amp;quot;belief&amp;quot; (3). Because both the algorithm (which based distinctions on translations) and the human annotators (who used WordNet senses) had low agreement in these cases, the WordNet sense distinctions may be overly fine-grained and, possibly, irrelevant to many language processing tasks.</Paragraph>
    <Paragraph position="10"> We continue to explore the viability of our method to automatically determine sense distinctions comparable to those achieved by human annotators. We are currently exploring methods to refine the clustering results as well as their comparison to results obtained from human annotators (e.g., the Gini Index [Boley, et al., 1999]).</Paragraph>
  </Section>
class="xml-element"></Paper>