<?xml version="1.0" standalone="yes"?>
<Paper uid="W06-2401">
  <Title>Named Entities Translation Based on Comparable Corpora</Title>
  <Section position="6" start_page="6" end_page="7" type="evalu">
    <SectionTitle>
5 Experiments
</SectionTitle>
    <Paragraph position="0"> As we have mentioned before, we have first extracted a set of 180 person, location and organization name-pairs from Euskaldunon Egunkaria 2002 newspaper and then we have translated them manually.</Paragraph>
    <Paragraph position="1"> We have used three evaluation measures to present the result of all the experiments: Precision = correctly translated NEsTranslated NEs Recall = correctly translated NEsAll NEs F score = 2 Precision RecallPrecision+Recall For the evaluation of the linguistic tool, we have used a parameter (x in the tables) which determines how many translation candidates will be used in each module at the most. This threshold is necessary since the output of both transliteration and arranging grammar is too big to work with in the next modules.</Paragraph>
    <Paragraph position="2"> The fr-min parameter in the tables specifies how often a candidate must occur in a data-set to be considered a likely NE translation proposal.</Paragraph>
    <Paragraph position="3"> fr. min -- x Precision Recall F-score  the linguistic tool, and searching its proposals in Google. If we observe these results taking into account the values of the x parameter, it seems that the bigger the x value is, the better results we get. But note that the best improvement is obtained when we use the maximum of 3 candidate instead of using just 1. We improved the system performance in 5%. While using 10 candidates, the performance increases in less than 1% compared to the results obtained when x value is 3.</Paragraph>
    <Paragraph position="4"> Regarding to the fr-min parameter, it seems that the best value is around 250. Moreover, duplicating this value, performance decreases. So we can say that when fr-min value exceeds 250, the system performs worse.</Paragraph>
    <Paragraph position="5"> For next comparatives, we will take the results given by the experiments using the values frmin=250 and x=1 as reference.</Paragraph>
    <Paragraph position="6"> When we search Wikipedia instead of Google (see Table 2), the system's recall decreases from 69.44% to 66.67%. This time the only searching restriction is that the candidate occurs at least once, and not n times. This is because the data-set offered by Wikipedia is significantly smaller than the one given by Google. Moreover, precision remains similar. So although it is a smaller data-set, Wikipedia seems to be similar to Google as far as the information significance of terms is concerned. fr. min -- x Precision Recall F-score  When we use the comparable corpus instead of the web, the linguistic tool performs a considerable enhancement in precision, a 13% improvement, but gets worse coverage. On the other hand, the language-independent tool achieves similar results with regard to the linguistic tool searching in the web. So the language-independent tool seems to be a good alternative for dealing with NE translation without no exhaustive linguistic work. Those results are detailed in Table 3.</Paragraph>
    <Paragraph position="7">  Finally, we have tried searching the proposals from the linguistic tool first in the comparable corpus. When no successful candidate is found in it, the system tries searching the web, in both Google and Wikipedia (See Table 4). In both experiments, precision is significantly lower than the one obtained when the system proposes candidates found in the comparable corpus, without no further search. However, the coverage increases in almost 5% in the trials carried out both with Google and Wikipedia. Therefore, the system's F-score  remains similar. Note that this time instead of performing better when Google is used, the searches done in Wikipedia give better results. Furthermore, the best results are obtained when combining comparable corpus and Wikipedia searches in the Linguistic tool.</Paragraph>
  </Section>
class="xml-element"></Paper>