<?xml version="1.0" standalone="yes"?>
<Paper uid="P04-1063">
  <Title>Multi-Engine Machine Translation with Voted Language Model</Title>
  <Section position="3" start_page="0" end_page="0" type="intro">
    <SectionTitle>
2 Confidence Models
</SectionTitle>
    <Paragraph position="0"> We take it here that the business of MEMT is about choosing among translation outputs from multiple MT systems, whether black box or not, for each input text. Therefore the question we want to address is, how do we go about choosing among MT outputs so that we end up with a best one? What we propose to do is to use some confidence models for translations generated by OTSs, and let them decide which one we should pick. We essentially work along the lines of Nomoto (2003). We review below some of the models proposed there, together with some motivation behind them.</Paragraph>
    <Paragraph position="1"> Confidence models he proposes come in two varieties: Fluency based model (FLM) and Alignment based model (ALM), which is actually an extension of FLM. Now suppose we have an English sentence e and its Japanese translation j generated by some OTS. (One note here: throughout the paper we work on English to Japanese translation.) FLM dictates that the quality of j as a translation of e be determined by:</Paragraph>
    <Paragraph position="3"> Pl(j) is the probability of j under a particular language model (LM) l.1 What FLM says is that the quality of a translation essentially depends on its log likelihood (or fluency) and has nothing to do with what it is a translation of.</Paragraph>
    <Paragraph position="4"> ALM extends FLM to include some information on fidelity. That is, it pays some attention to how faithful a translation is to its source text. ALM does this by using alignment models from the statistical machine translation literature (Brown et al., 1993).</Paragraph>
    <Paragraph position="5"> Here is what ALM looks like.</Paragraph>
    <Paragraph position="7"> Model 1. ALM takes into account the fluency of a translation output (given by Pl(j)) and the degree of association between e and j (given by Q(e  |j)), which are in fact two features generally agreed in the MT literature to be most relevant for assessing the quality of translations (White, 2001).</Paragraph>
    <Paragraph position="8"> One problem with FLM and ALM is that they fail to take into account the reliability of an OTS system. As Nomoto (2003) argues, it is reasonable to believe that some MT systems could inherently be more prone to error and outputs they produce tend to be of less quality than those from other systems, no matter what the outputs' fluency or translation probability may be. ALM and FLM work solely on statistical information that can be gathered from source and target sentences, dismissing any operational bias that an OTS might have on a particular task.</Paragraph>
    <Paragraph position="9"> Nomoto (2003) responds to the problem by introducing a particular regression model known as Support Vector regression (SVR), which enables him to exploit bias in performance of OTSs. What SVR is intended to do is to modify confidence scores FLM and ALM produce for MT outputs in such a way that they may more accurately reflect their independent evaluation involving human translations or judgments. SVR is a multi-dimensional regressor, and works pretty much like its enormously popular counterpart, Support Vector classification, except that we are going to work with real numbers for target values and construct the margin, using Vapnik's epsilon1-insensitive loss function (Sch&amp;quot;olkopf et al., 1998).</Paragraph>
    <Paragraph position="11"> 1***wm. Assume a uniform prior for l.</Paragraph>
    <Paragraph position="12"> SVR looks something like this.</Paragraph>
    <Paragraph position="13"> h(vectorx) = vectorw*vectorx+b, with input data vectorx = (x1,...,xm) and the corresponding weights vectorw = (w1,...,wm). 'x * y' denotes the inner product of x and y. vectorx could be a set of features associated with e and j. Parameters vectorw and b are something determined by SVR.</Paragraph>
    <Paragraph position="14"> It is straightforward to extend the ALM and FLM with SVR, which merely consists of plugging in either model as an input variable in the regressor. This would give us the following two SVR models with</Paragraph>
    <Paragraph position="16"> Notice that h(*) here is supposed to relate FLM or ALM to some independent evaluation metric such as BLEU (Papineni et al., 2002), not the log likelihood of a translation.</Paragraph>
    <Paragraph position="17"> With confidence models in place, define a MEMT model Ps by:</Paragraph>
    <Paragraph position="19"> Here e represents a source sentence, J a set of translations for e generated by OTSs, and th denotes some confidence model under an LM l. Throughout the rest of the paper, we let FLMps and ALMps denote MEMT systems based on FLM and ALM, respectively, and similarly for others.</Paragraph>
  </Section>
class="xml-element"></Paper>