<?xml version="1.0" standalone="yes"?>
<Paper uid="W06-2505">
  <Title>Multilingual versus Monolingual WSD</Title>
  <Section position="7" start_page="37" end_page="38" type="evalu">
    <SectionTitle>
5 Results and discussion
</SectionTitle>
    <Paragraph position="0"> Table 4 presents the number of different sentences analyzed for each of the verbs (after grouping and eliminating the repeated sentences), the English (E) senses and (nonsynonyms) Portuguese (P) translations in our corpus, followed by the percentage of occurrences of each of the categories outlined in Section 4 (a - e) with respect to the number of senses (# Senses) for that verb. Items (c) and (d) were grouped, since for practical purposes it is not important to tell if the P word translating the various E senses encompasses one or many senses. For items (b) and (c&amp;d) we also present the average of P translations per E sense ((b) average), and the average of E senses per P translation, respectively ((c&amp;d) average).</Paragraph>
    <Paragraph position="1"> We divided the analysis of these results according to our two cases (cf. Section 4): the first covers items (c&amp;d) and (e) (light grey in Table 4), while the second covers items (a) and (b) (dark grey in Table 4).</Paragraph>
    <Paragraph position="2"> 1) Items (c), (d) and (e): n senses - ? translation(s) null The number of senses in the corpus is almost always greater than the number of translations, suggesting that the level of sense distinctions in WordNet can be too fine-grained for translation applications The numbers of senses and translations are in an opposite relation comparing to the one shown in Table 3, where the number of possible translations was larger than the number of possible senses. This shows that indeed many of the possible translations are synonyms.</Paragraph>
    <Paragraph position="3"> On average, the level of ambiguity decreased from 40.3 (possible senses) to 24.4 (possible translations), if the monolingual and multilingual ambiguity are compared in the corpus. If we consider the five most ambiguous verbs, the level of ambiguity decreased from 58.8 to 35. For the other three less ambiguous verbs, the level of ambiguity decreased from 9.3 to 6.7.</Paragraph>
    <Paragraph position="4"> Column % (c&amp;d) shows the percentage of senses, with respect to the total shown in the third column (# Senses), which share translations with other senses. A shared translation means that several senses of the verb have the same translation. (c&amp;d) average indicates the average number of E senses per P translation, for those cases where translations are shared. For all verbs, on average translations cover more than two senses. The level of variation in the number of shared translations among senses is high, e.g., from 2 (translation = &amp;quot;organizar&amp;quot;) to 27 (translation = &amp;quot;dar&amp;quot;) for the verb &amp;quot;to give&amp;quot;. Contrasting the percentage of senses that share translations, in % (c), with the percentages in % (d), which refers to the senses for which translations are not shared, we can see that the great majority of senses have translations in common with other senses, and thus the disambiguation among these senses would not be necessary in most of the cases. In fact, it could result in errors, since an incorrect sense could be chosen.</Paragraph>
    <Paragraph position="5"> 2) Items (a) and (b): 1 sense - ? translation(s) As previously mentioned, the differences in the sense inventory for monolingual and multilingual WSD are not only due to the fact that sense distinctions in WordNet are too refined. That would only indicate that using monolingual WSD for multilingual purposes implies unnecessary work.</Paragraph>
    <Paragraph position="6"> However, we consider that the most important problem is the one evidenced by item (b) in the sixth column in Table 4. For all the verbs except &amp;quot;to ask&amp;quot; (the least ambiguous), there were cases in which different occurrences of the same sense were translated into different, non-synonyms words. Although the proportion of senses with only one translation is greater, as shown by item (a) in the fifth column, the percentage of senses with more than one translation is impressive, especially for the five most ambiguous verbs. In face of this, the lack of disambiguation of a word during translation based on the fact that the word is not ambiguous in the source language can result in very serious translation errors when monolingual methods are employed for multilingual WSD. Therefore, this also shows that, for these verbs, sense inventories that are specific to the translation between the pair of languages under consideration would be more appropriate to achieve effective WSD.</Paragraph>
    <Section position="1" start_page="37" end_page="38" type="sub_section">
      <SectionTitle>
5.1 Agreement between translators
</SectionTitle>
      <Paragraph position="0"> In an attempt to quantify the agreement between the two groups of translators, we computed the Kappa coefficient for annotation tasks, as defined by Carletta (1996). Kappa was calculated separately for our two areas of inquiry, i.e., cases (1) and (2) discussed in Section 5.</Paragraph>
      <Paragraph position="1"> In the experiment referring to case (1), groups were considered to agree about a sense of a verb if they both judged that the translation of such  verb was or was not shared by other senses. For example, both groups agreed that the word &amp;quot;fazer&amp;quot; should be used to translate occurrences of many senses of the verb &amp;quot;to make&amp;quot;, including &amp;quot;engage in&amp;quot;, &amp;quot;give certain properties to something&amp;quot;, and &amp;quot;make or cause to be or to become&amp;quot;. On the other hand, the groups disagreed about the sense &amp;quot;go off or discharge&amp;quot; of the phrasal verb &amp;quot;to go off&amp;quot;: the first group found that the translation of that sense, &amp;quot;disparar&amp;quot;, did not refer to any other sense, while the second group used that word to translate also the sense &amp;quot;be discharged or activated&amp;quot; of the same phrasal verb. In the experiment with case (2), groups were considered to agree about a sense if they both judged that the sense had or had not more than one translation. For example, both groups agreed that the sense &amp;quot;reach a state, relation, or condition&amp;quot; of the verb &amp;quot;to come&amp;quot; should be translated by more than one Portuguese word, including &amp;quot;terminar&amp;quot;, &amp;quot;vir&amp;quot;, and &amp;quot;chegar&amp;quot;. They also agreed that the sense &amp;quot;move toward, travel toward something or somebody or approach something or somebody&amp;quot; of the same verb had only one translation, namely &amp;quot;vir&amp;quot;.</Paragraph>
      <Paragraph position="2"> The average Kappa coefficient obtained was 0.66 for item (1), and 0.65 for item (2). There is not a reference value for this particular annotation task (translation annotation), but the levels of agreement pointed by Kappa here can be considered satisfactory. The agreement levels are close to the coefficient suggested by Carletta as indicative of a good agreement level for discourse annotation (0.67), and which has been adopted as a cutoff in Computational Linguistics.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>