<?xml version="1.0" standalone="yes"?>
<Paper uid="W02-0805">
  <Title>Polysemy and Sense Proximity in the Senseval-2 Test Suite. Irina Chugur irina@lsi.uned.es</Title>
  <Section position="4" start_page="0" end_page="0" type="metho">
    <SectionTitle>
4 Experiment design
</SectionTitle>
    <Paragraph position="0"> Following (Resnik and Yarowsky, 1999), we carried out an experiment based on free annotations with the first preferred translation of Senseval-2 nouns and adjectives in hand-disambiguated contexts.</Paragraph>
    <Paragraph position="1"> 11 native or bilingual speakers of 4 languages1 with a level of English proficiency from medium to 1This main set of languages includes Bulgarian, Russian, Spanish and Urdu.</Paragraph>
    <Paragraph position="2"> high were asked to translate marked words in context into their mother tongue.</Paragraph>
    <Paragraph position="3"> As working material, we used part of the Senseval-2 data. Whenever possible, we selected three contexts (with the highest inter annotator agreement) for each of the 182 noun and adjective senses in the Senseval-2 English test suite. However, for 16 senses there was only one instance in the corpus and 6 senses had only two occurrences. The final data set was composed of 508 short contexts for 182 senses of 44 words of the Senseval-2 test suite. These instances, randomly ordered, were presented to annotators without sense tag, so that each tagger had to deduce the sense of the marked word from the context and type its first preferred translation into his language in the answer line. This is an example of the input for annotators: fine 40129 Mountains on the other side of the valley rose from the mist like islands, and here and there flecks of cloud, as pale and a83 taga84 finea83 /taga84 as sea-spray, trailed across their sombre, wooded slopes.</Paragraph>
    <Paragraph position="4"> ANSWER: * * The collected data was used to compute proximity for every sense pair a1a3a2a12a11a13a1a5a4 in the sample. Stability was computed using the same data, for all senses in the sample except for 16 cases which had one instance in the corpus.</Paragraph>
    <Paragraph position="5"> In order to evaluate how using a more extensive and more varied set of languages can affect the results of the experiment, we selected a smaller control subset of 65 instances of 23 senses corresponding to 3 nouns and 2 adjectives. Annotations for this subset were collected from 29 speakers of 12 languages 2 covering 5 language families.</Paragraph>
  </Section>
class="xml-element"></Paper>