<?xml version="1.0" standalone="yes"?>
<Paper uid="W06-1902">
  <Title>The Affect of Machine Translation on the Performance of Arabic- English QA System</Title>
  <Section position="3" start_page="9" end_page="9" type="intro">
    <SectionTitle>
2 Related Research
</SectionTitle>
    <Paragraph position="0"> CLIR is an active area, extensive research on CLIR and the effects of MT on QA systems' retrieval effectiveness has been conducted. Lin and Mitamua (2004) point out that the quality of translation is fully dependent upon the MT system employed.</Paragraph>
    <Paragraph position="1"> Perret (2004) proposed a question answering system designed to search French documents in response to French queries. He used automatic translation resources to translate the original queries from (Dutch, German, Italian, Portuguese, Spanish, English and Bulgarian) and reports the performance level in the monolingual task was 24.5% dropping to 17% in the bilingual task. A similar experiment was conducted by Plamondon and Foster (2003) on TREC questions and measured a drop of 44%, and in another experiment using Babelfish, the performance dropped even more, 53%. They believe that CLEF questions were easier to process because they did not include definition questions, which are harder to translate. Furthermore, Plamondon and Foster (2004) compare the cross language version of their Quantum QA system with the monolingual English version on CLEF questions and note the performance of a cross-language system (French questions and English documents) was 28% lower than the monolingual system using IBM1 translation.</Paragraph>
    <Paragraph position="2"> Tanev et al. (2004) note that DIOGENE system, which relies on the Multi-WordNet, performs 15% better in the monolingual (Italian-Italian) than cross-language task (Italian-English). In Magnini et al.'s (2004) report for the year 2003, the average performance of correct answers on monolingual tasks was 41% and 25% in the bilingual tasks. In addition in the year 2004, the average accuracy in the monolingual tasks was 23.7% and 14.7% in bilingual tasks.</Paragraph>
    <Paragraph position="3"> As elucidated above, much research has been conducted to evaluate the effectiveness of QA systems in a cross language platform by employing MT systems to translate the queries from the source language to the target language. However, most of them are focused on European language pairs. To our knowledge, only one past example of research has investigated the performance of a cross-language Arabic-English QA system Rosso et al (2005). The QA system used by Rosso et al (2005) is based on a system reported in Del Castillo (2004). Their experiment was carried out using the question corpus of the CLEF-2003 competition. They used questions in English and compared the answers with those obtained after the translation back into English from an Arabic question corpus which was manually translated. For the Arabic-English translation process, an automatic machine translator, the TARJIM Arabic-English machine translation system, was used. Rosso el al reported a decrease of QA accuracy by more than 30% which was caused by the translation process. null Work in the Rosso paper was limited to a single QA and MT system and also did not analyze types of errors or how those errors affected different types of QA questions. Therefore, it was decided to conduct further research on MT systems and its affect on the performance in QA systems. This paper presents an extension on the previous mentioned study, but with more diverse ranges of TREC data set using different QA system and different MT system.</Paragraph>
  </Section>
class="xml-element"></Paper>