<?xml version="1.0" standalone="yes"?>
<Paper uid="C02-1072">
  <Title>A Comparative Evaluation of Data-driven Models in Translation Selection of Machine Translation</Title>
  <Section position="4" start_page="0" end_page="0" type="evalu">
    <SectionTitle>
4 Experiment and Evaluation
</SectionTitle>
    <Paragraph position="0"/>
    <Section position="1" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
4.1 Data for Latent Space and
Dictionary
</SectionTitle>
      <Paragraph position="0"> In the experiment, we used two kinds of corpus data, one for constructing LSA and PLSA spaces and the other for building a dictionary containing grammatical relations and a test set.</Paragraph>
      <Paragraph position="1"> 79,919 texts in 1988 AP news corpus from</Paragraph>
      <Paragraph position="3"> + Training { For each training example hx;f(x)i, add the example to the list training examples.</Paragraph>
      <Paragraph position="4"> + Classiflcation { Given a query instance xq to be classifled, null / Let x1;:::;xk denote the k instances from training examples that are nearest to xq.</Paragraph>
      <Paragraph position="5"> / Return</Paragraph>
      <Paragraph position="7"> are extracted. We built 200 dimensions in SVD ofLSAand128latentdimensionsofPLSA.The difierence of the numbers was caused from the degree of computational complexity in learning phase. Actually, PLSA of 128 latent factors required 50-fold time as much as LSA hiring 200 eigen-vectorspaceduringbuildinglatentspaces.</Paragraph>
      <Paragraph position="8"> This was caused by 50 iterations which made  gle vector lanczos algorithm derived from SVD-PACK when constructing LSA space. (Berry et al., 1993). We generated both of LSA and PLSA spaces, with each word having a vector of 200 and 128 dimensions, respectively. The similarity of any two words could be estimated by performing cosine computation between two vectors representing coordinates of the words in the spaces.</Paragraph>
      <Paragraph position="9"> Table 4 shows 5 most similar words of randomly selected words from 3,443 examples. We extracted 3,443 example sentences containing grammatical relations, like verb-object, subject-verb and adjective-noun, from Wall Street Journal corpus of 220,047 sentences and other newspapers corpus of 41,750 sentences, totally 261,797 sentences. We evaluated the accuracy performance of each grammatical relation. 2,437, 188, and 818 examples were utilized for verb-object, subject-verb, and adjectivenoun, respectively. The selection accuracy was measured using 5-fold cross validation for each grammatical relation. Sample sentences of each grammatical relation were divided into flve disjoint samples and each sample became a test sample once in the experiment and the remaining four samples were combined to make up a collocation dictionary.</Paragraph>
    </Section>
    <Section position="2" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
4.2 Experimental Result
</SectionTitle>
      <Paragraph position="0"> Table 5 and flgure 1-3 show the results of translation selection with respect to the applied model and to the value of k. As shown in Table 5, similarity based on data-driven model could improve the selection accuracy up to 20% as  contrasted with the direct matching method. We could obtain the result that PLSA could improve the accuracy more than LSA in almost all cases. The amount of improvement is varied from -0.12% to 2.96%.</Paragraph>
      <Paragraph position="1"> As flgure 1-3 show, the value of k had afiection to the translation accuracy in PLSA, however, not in LSA. From this, we could not declare whether the value of k and translation accuracy have relationship of each other or not in the data-driven models described in this paper. However,wecouldalsoflndthatthedegree of accuracy was raised in accordance with the value of k in PLSA. From this, we consequently inferred that the latent semantic space generatedbyPLSAhadmoresounddistributionwith null re ection of well-structured semantic structure  LSA, and PLSA. The words are stems of original words. The flrst row of each selected word stands for the most similar words in LSA semantic space and the second row stands for those in the PLSA space.</Paragraph>
      <Paragraph position="2"> selected words most similar words plant westinghous isocyan shutdown zinc manur radioact hanford irradi tritium biodegrad car buick oldsmobil chevrolet sedan corolla highwai volkswagen sedan vehicular vehicle home parapleg broccoli coconut liverpool jamal memori baxter hanlei corwin headston business entrepreneur corpor custom ventur flrm digit compat softwar blackston zayr ship vessel sail seamen sank sailor destroy frogmen maritim skipper vessel Table5: Translationaccuracyinvariouscase. Theflrstcolumnstandsforeachgrammaticalrelation and the second column stands for the used models, LSA or PLSA. And other three columns stand for the accuracy ratio (rm) with respect to the value of k. The numbers in parenthesis of the flrst column show the translation accuracy ratio of simple dictionary search method (rs). And numbers in the other parenthesis were obtained by rm Y=rs.</Paragraph>
      <Paragraph position="3"> grammatical used k = 1 k = 5 k = 10  lations, subj-verb, showed an exceptional case, which seemed to be caused by the small size of examples, 188.</Paragraph>
      <Paragraph position="4"> Selection errors taking place in LSA and PLSA models were caused mainly by the following reasons. First of all, the size of vocabulary should be limited by computation complexity. In this experiment, we acquired below 20,000 words for the vocabulary, which could not cover a section of corpus data. Second, the stemming algorithm was not robust for an indexing. For example, 'house' and 'housing' are regarded as a same word as 'hous'. This fact broughtabouthardnessinre ectingthesemantic structure more precisely. And flnally, the meaning of similar word is somewhat varied in the machine translation fleld and the information retrieval fleld. The selectional restriction tends to depend a little more upon semantic type like human-being, place and etc., than on the context in a document.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>