<?xml version="1.0" standalone="yes"?>
<Paper uid="P06-1097">
  <Title>Sydney, July 2006. c(c)2006 Association for Computational Linguistics Semi-Supervised Training for Statistical Word Alignment</Title>
  <Section position="4" start_page="0" end_page="769" type="metho">
    <SectionTitle>
2 Data Sets and Baseline
</SectionTitle>
    <Paragraph position="0"> We conduct experiments on alignment and translation tasks using Arabic/English and French/English data sets (see Table 1 for details). Both sets have training data and two gold standard word alignments for small samples of the training data, which we use as the alignment  and the alignment test set sub-corpora, separated by /. discriminative training set and alignment test set. Translation quality is evaluated by translating a held-out translation test set. An additional translation set called the Maximum BLEU set is employed by the SMT system to train the weights associated with the components of its log-linear model (Och, 2003).</Paragraph>
    <Paragraph position="1"> The training corpora are publicly available: both the Arabic/English data and the French/English Hansards were released by LDC. We created the manual word alignments ourselves, following the Blinker guidelines (Melamed, 1998).</Paragraph>
    <Paragraph position="2"> To train our baseline systems we follow a standard procedure. The models were trained two times, rst using French or Arabic as the source language and then using English as the source language. For each training direction, we run GIZA++ (Och and Ney, 2003), specifying 5 iterations of Model 1, 4 iterations of the HMM model (Vogel et al., 1996), and 4 iterations of Model 4. We quantify the quality of the resulting hypothesized alignments with F-measure using the manually aligned sets.</Paragraph>
    <Paragraph position="3"> We present the results for three different conditions in Table 2. For the F to E direction the models assign non-zero probability to alignments consisting of links from one Foreign word to zero or more English words, while for E to F the models assign non-zero probability to alignments consisting of links from one English word to zero or more Foreign words. It is standard practice to improve the nal alignments by combining the F to E and E to F directions using symmetrization heuristics. We use the union , re ned and intersection heuristics de ned in (Och and Ney, 2003) which are used in conjunction with IBM Model 4 as the baseline in virtually all recent work on word alignment. In Table 2, we report the best symmetrized results.</Paragraph>
    <Paragraph position="4"> The low F-measure scores of the baselines motivate our work.</Paragraph>
  </Section>
  <Section position="5" start_page="769" end_page="771" type="metho">
    <SectionTitle>
3 Improving Word Alignments
</SectionTitle>
    <Paragraph position="0"/>
    <Section position="1" start_page="769" end_page="771" type="sub_section">
      <SectionTitle>
3.1 Discriminative Reranking of the IBM
Models
</SectionTitle>
      <Paragraph position="0"> We reinterpret the ve groups of parameters of Model 4 listed in the rst ve lines of Table 3 as sub-models of a log-linear model (see Equation 1).</Paragraph>
      <Paragraph position="1"> Each sub-model hm has an associated weight lm.</Paragraph>
      <Paragraph position="2"> Given a vector of these weights l, the alignment search problem, i.e. the search to return the best alignment ^a of the sentences e and f according to the model, is speci ed by Equation 2.</Paragraph>
      <Paragraph position="4"> from a particular e 12 translation table from HMM iteration 4 5 d&gt;1(trianglej) movement probs of other Foreign words translated from a particular e 13 zero fertility English word penalty 6 translation table from re ned combination of both alignments 14 non-zero fertility English word penalty 7 translation table from union of both alignments 15 NULL Foreign word penalty 8 translation table from intersection of both alignments 16 non-NULL Foreign word penalty  Log-linear models are often trained to maximize entropy, but we will train our model directly on the nal performance criterion. We use 1[?]F-measure as our error function, comparing hypothesized word alignments for the discriminative training set with the gold standard.</Paragraph>
      <Paragraph position="5"> Och (2003) has described an ef cient exact one-dimensional error minimization technique for a similar search problem in machine translation. The technique involves calculating a piecewise constant function fm(x) which evaluates the error of the hypotheses which would be picked by equation 2 from a set of hypotheses if we hold all weights constant, except for the weight lm (which is set to x).</Paragraph>
      <Paragraph position="6"> The discriminative reranking algorithm is initialized with the parameters of the sub-models th, an initial choice of the l vector, gold standard word alignments (labels) for the alignment discriminative training set, the constant N specifying  the N-best list size used1, and an empty master set of hypothesized alignments. The algorithm is a three step loop: 1. Enrich the master set of hypothesized alignments by producing an N-best list using l. If all of the hypotheses in the N-best list are already in the master set, the algorithm has converged, so terminate the loop.</Paragraph>
      <Paragraph position="7"> 2. Consider the current l vector and 999 additional randomly generated vectors, setting l to the vector with lowest error on the master set.</Paragraph>
      <Paragraph position="8"> 3. Repeatedly run Och's one-dimensional error minimization step until there is no further error reduction (this results in a new vector l). 1N = 128 for our experiments 3.2 Improvements to the Model and Search  We de ne new sub-models to model factors not captured by Model 4. These are lines 6 to 16 of Table 3, where we use the E to F alignment direction as an example. We use word-level translation tables informed by both the E to F and the F to E translation directions derived using the three symmetrization heuristics, the E to F translation table from the nal iteration of the HMM model and an E to F translation table derived using approximative stemming. The approximative stemming sub-model (sub-model 9) uses the rst 4 letters of each vocabulary item as the stem for English and French while for Arabic we use the full word as the stem. We also use sub-models for backed off fertility, and direct penalization of unaligned English words ( zero fertility ) and aligned English words, and unaligned Foreign words ( NULL-generated words) and aligned Foreign words. This is a small sampling of the kinds of knowledge sources we can use in this framework; many others have been proposed in the literature.</Paragraph>
      <Paragraph position="9"> Table 4 shows an evaluation of discriminative reranking. We observe:  1. The rst line is the starting point, which is the Viterbi alignment of the 4th iteration of HMM training.</Paragraph>
      <Paragraph position="10"> 2. The 1-to-many alignments generated by discriminatively reranking Model 4 are better than the 1-to-many alignments of four iterations of Model 4.</Paragraph>
      <Paragraph position="11"> 3. The 1-to-many alignments of the discrimina- null tively reranked extended model are much better than four iterations of Model 4.</Paragraph>
      <Paragraph position="12">  symmetrization, but some of the gain is lost as we are optimizing the F-measure of the 1-to-many alignments rather than the F-measure of the many-to-many alignments directly.</Paragraph>
      <Paragraph position="13"> Overall, the results show our approach is better than or competitive with running four iterations of unsupervised Model 4 training.</Paragraph>
      <Paragraph position="14">  Brown et al. (1993) introduced operations de ning a hillclimbing search appropriate for Model 4. Their search starts with a complete hypothesis and exhaustively applies two operations to it, selecting the best improved hypothesis it can nd (or terminating if no improved hypothesis is found). This search makes many search errors2. We developed a new alignment algorithm to reduce search errors: * We perform an initial hillclimbing search (as in the baseline algorithm) but construct a priority queue of possible other candidate alignments to consider.</Paragraph>
      <Paragraph position="15"> * Alignments which are expanded are marked so that they will not be returned to at a future point in the search.</Paragraph>
      <Paragraph position="16"> * The alignment search operates by considering complete hypotheses so it is an anytime algorithm (meaning that it always has a current best guess). Timers can therefore be used to terminate the processing of the priority queue of candidate alignments.</Paragraph>
      <Paragraph position="17"> The rst two improvements are related to the well-known Tabu local search algorithm (Glover, 2A search error in a word aligner is a failure to nd the best alignment according to the model, i.e. in our case a failure to maximize Equation 2.</Paragraph>
      <Paragraph position="18"> 1986). The third improvement is important for restricting total time used when producing alignments for large training corpora.</Paragraph>
      <Paragraph position="19"> We performed two experiments. The rst evaluates the number of search errors. For each corpus we sampled 1000 sentence pairs randomly, with no sentence length restriction. Model 4 parameters are estimated from the nal HMM Viterbi alignment of these sentence pairs. We then search to try to nd the Model 4 Viterbi alignment with both the new and old algorithms, allowing them both to process for the same amount of time. The percentage of known search errors is the percentage of sentences from our sample in which we were able to nd a more probable candidate by applying our new algorithm using 24 hours of computation for just the 1000 sample sentences. Table 5 presents the results, showing that our new algorithm reduced search errors in all cases, but further reduction could be obtained. The second experiment shows the impact of the new search on discriminative reranking of Model 4 (see Table 6). Reduced search errors lead to a better t of the discriminative training corpus.</Paragraph>
    </Section>
  </Section>
  <Section position="6" start_page="771" end_page="773" type="metho">
    <SectionTitle>
4 Semi-Supervised Training for Word
Alignments
</SectionTitle>
    <Paragraph position="0"> Intuitively, in approximate EM training for Model 4 (Brown et al., 1993), the E-step corresponds to calculating the probability of all alignments according to the current model estimate, while the M-step is the creation of a new model estimate given a probability distribution over alignments (calculated in the E-step).</Paragraph>
    <Paragraph position="1"> In the E-step ideally all possible alignments should be enumerated and labeled with p(a|e, f), but this is intractable. For the M-step, we would like to count over all possible alignments for each sentence pair, weighted by their probability according to the model estimated at the previous  step. Because this is not tractable, we make the assumption that the single assumed Viterbi alignment can be used to update our estimate in the Mstep. This approximation is called Viterbi training. Neal and Hinton (1998) analyze approximate EM training and motivate this type of variant.</Paragraph>
    <Paragraph position="2"> We extend approximate EM training to perform a new type of training which we call Minimum Error / Maximum Likelihood Training. The intuition behind this approach to semi-supervised training is that we wish to obtain the advantages of both discriminative training (error minimization) and approximate EM (which allows us to estimate a large numbers of parameters even though we have very few gold standard word alignments). We introduce the EMD algorithm, in which discriminative training is used to control the contributions of sub-models (thereby minimizing error), while a procedure similar to one step of approximate EM is used to estimate the large number of sub-model parameters.</Paragraph>
    <Paragraph position="3"> A brief sketch of the EMD algorithm applied to our extended model is presented in Figure 1.</Paragraph>
    <Paragraph position="4"> Parameters have a superscript t representing their value at iteration t. We initialize the algorithm with the gold standard word alignments (labels) of the word alignment discriminative training set, an initial l, N, and the starting alignments (the iteration 4 HMM Viterbi alignment). In line 2, we make iteration 0 estimates of the 5 sub-models of Model 4 and the 6 heuristic sub-models which are iteration dependent. In line 3, we run discriminative training using the algorithm from Section 3.1. In line 4, we measure the error of the resulting l vector. In the main loop in line 7 we align the full training set (similar to the E-step of EM), while in line 8 we estimate the iteration-dependent sub-models (similar to the M-step of EM). Then  1: Algorithm EMD(labels, lprime, N, starting alignments) 2: estimate th0m for m = 1 to 11 3: l0 = Discrim(th0, lprime, labels, N) 4: e0 = E(l0, labels) 5: t = 1 6: loop 7: align full training set using lt[?]1 and tht[?]1m 8: estimate thtm for m = 1 to 11 9: lt = Discrim(tht, lprimeprime, labels, N) 10: et = E(lt, labels) 11: if et &gt;= et[?]1 then 12: terminate loop 13: end if 14: t = t + 1 15: end loop 16: return hypothesized alignments of full training set  we perform discriminative reranking in line 9 and check for convergence in lines 10 and 11 (convergence means that error was not decreased from the  1. In both cases there is improved F-measure on the second iteration of semi-supervised training, indicating that the EMD algorithm performs better than one step discriminative reranking.</Paragraph>
    <Paragraph position="5"> 2. The French/English data set has converged3 after the second iteration.</Paragraph>
    <Paragraph position="6"> 3. The Arabic/English data set converged after  improvement for the rst, second and third iterations.</Paragraph>
    <Paragraph position="7"> We also performed an additional experiment for French/English aimed at understanding the potential contribution of the word aligned data without  the new algorithm4. Like Ittycheriah and Roukos (2005), we converted the alignment discriminative training corpus links into a special corpus consisting of parallel sentences where each sentence consists only of a single word involved in the link. We found that the information in the links was washed out by the rest of the data and resulted in no change in the alignment test set's F-Measure. Callison-Burch et al. (2004) showed in their work on combining alignments of lower and higher quality that the alignments of higher quality should be given a much higher weight than the lower quality alignments. Using this insight, we found that adding 10,000 copies of the special corpus to our training data resulted in the highest alignment test set gain, which was a small gain of 0.6 F-Measure. This result suggests that while the link information is useful for improving F-Measure, our improved methods for training are producing much larger improvements.</Paragraph>
  </Section>
  <Section position="7" start_page="773" end_page="774" type="metho">
    <SectionTitle>
5 Improvement of MT Quality
</SectionTitle>
    <Paragraph position="0"> The symmetrized alignments from the last iteration of EMD were used to build phrasal SMT systems, as were the symmetrized Model 4 alignments (the baseline). Aside from the nal alignment, all other resources were held constant between the baseline and contrastive SMT systems, including those based on lower level alignments models such as IBM Model 1. For all of our experiments, we use two language models, one built using the English portion of the training data and the other built using additional English news data.</Paragraph>
    <Paragraph position="1"> We run Maximum BLEU (Och, 2003) for 25 iterations individually for each system.</Paragraph>
    <Paragraph position="2"> Table 8 shows our results. We report BLEU (Papineni et al., 2001) multiplied by 100. We also show the F-measure after heuristic symmetrization of the alignment test sets. The table shows that 4We would like to thank an anonymous reviewer for suggesting that this experiment would be useful even when using a small discriminative training corpus.</Paragraph>
    <Paragraph position="3"> our algorithm produces heuristically symmetrized nal alignments of improved F-measure. Using these alignments in our phrasal SMT system, we produced a statistically signi cant BLEU improvement (at a 95% con dence interval a gain of 0.78 is necessary) on the French/English task and a statistically signi cant BLEU improvement on the Arabic/English task (at a 95% con dence interval a gain of 1.2 is necessary).</Paragraph>
    <Section position="1" start_page="773" end_page="774" type="sub_section">
      <SectionTitle>
5.1 Error Criterion
</SectionTitle>
      <Paragraph position="0"> The error criterion we used for all experiments is 1 [?] F-measure. The formula for F-measure is shown in Equation 3. (Fraser and Marcu, 2006) established that tuning the trade-off between Precision and Recall in the F-Measure formula will lead to the best BLEU results. We tuned a by building a collection of alignments using our baseline system, measuring Precision and Recall against the alignment discriminative training set, building SMT systems and measuring resulting BLEU scores, and then searching for an appropriate a setting. We searched a = 0.1, 0.2, ..., 0.9 and set a so that the resulting F-measure tracks BLEU to the best extent possible. The best settings were a = 0.2 for Arabic/English and a = 0.7 for French/English, and these settings of a were used for every result reported in this paper. See (Fraser and Marcu, 2006) for further details.</Paragraph>
      <Paragraph position="1">  Previous work on discriminative training for word-alignment differed most strongly from our approach in that it generally views word-alignment as a supervised task. Examples of this perspective include (Liu et al., 2005; Ittycheriah and Roukos, 2005; Moore, 2005; Taskar et al., 2005). All of these also used knowledge from one of the IBM Models in order to obtain competitive results  with the baseline (with the exception of (Moore, 2005)). We interleave discriminative training with EM and are therefore performing semi-supervised training. We show that semi-supervised training leads to better word alignments than running unsupervised training followed by discriminative training. null Another important difference with previous work is that we are concerned with generating many-to-many word alignments. Cherry and Lin (2003) and Taskar et al. (2005) compared their results with Model 4 using intersection by looking at AER (with the Sure versus Possible link distinction), and restricted themselves to considering 1-to-1 alignments. However, union and rened alignments, which are many-to-many, are what are used to build competitive phrasal SMT systems, because intersection performs poorly, despite having been shown to have the best AER scores for the French/English corpus we are using (Och and Ney, 2003). (Fraser and Marcu, 2006) recently found serious problems with AER both empirically and analytically, which explains why optimizing AER frequently results in poor machine translation performance.</Paragraph>
      <Paragraph position="2"> Finally, we show better MT results by using F-measure with a tuned a value. The only previous discriminative approach which has been shown to produce translations of similar or better quality to those produced by the symmetrized baseline was (Ittycheriah and Roukos, 2005). They had access to 5000 gold standard word alignments, considerably more than the 100 or 110 gold standard word alignments used here. They also invested significant effort in sub-model engineering (producing both sub-models speci c to Arabic/English alignment and sub-models which would be useful for other language pairs), while we use sub-models which are simple extensions of Model 4 and language independent.</Paragraph>
      <Paragraph position="3"> The problem of semi-supervised learning is often de ned as using unlabeled data to help supervised learning (Seeger, 2000). Most work on semi-supervised learning uses underlying distributions with a relatively small number of parameters. An initial model is estimated in a supervised fashion using the labeled data, and this supervised model is used to attach labels (or a probability distribution over labels) to the unlabeled data, then a new supervised model is estimated, and this is iterated. If these techniques are applied when there are a small number of labels in relation to the number of parameters used, they will suffer from the overcon dent pseudo-labeling problem (Seeger, 2000), where the initial labels of poor quality assigned to the unlabeled data will dominate the model estimated in the M-step. However, there are tasks with large numbers of parameters where there are suf cient labels. Nigam et al. (2000) addressed a text classi cation task. They estimate a Naive Bayes classi er over the labeled data and use it to provide initial MAP estimates for unlabeled documents, followed by EM to further rene the model. Callison-Burch et al. (2004) examined the issue of semi-supervised training for word alignment, but under a scenario where they simulated suf cient gold standard word alignments to follow an approach similar to Nigam et al. (2000).</Paragraph>
      <Paragraph position="4"> We do not have enough labels for this approach.</Paragraph>
      <Paragraph position="5"> We are aware of two approaches to semi-supervised learning which are more similar in spirit to ours. Ivanov et al. (2001) used discriminative training in a reinforcement learning context in a similar way to our adding of a discriminative training step to an unsupervised context. A large body of work uses semi-supervised learning for clustering by imposing constraints on clusters. For instance, in (Basu et al., 2004), the clustering system was supplied with pairs of instances labeled as belonging to the same or different clusters.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>