<?xml version="1.0" standalone="yes"?>
<Paper uid="W06-3602">
  <Title>Efficient Dynamic Programming Search Algorithms for Phrase-Based SMT</Title>
  <Section position="2" start_page="0" end_page="9" type="intro">
    <SectionTitle>
1 Introduction
</SectionTitle>
    <Paragraph position="0"> This paper deals with dynamic programming based decoding and alignment algorithms for phrase-based SMT.</Paragraph>
    <Paragraph position="1"> Dynamic Programming based search algorithms are being used in speech recognition (Jelinek, 1998; Ney et al., 1992) as well as in statistical machine translation (Tillmann et al., 1997; Niessen et al., 1998; Tillmann and Ney, 2003). Here, the decoding algorithms are described as shortest path finding algorithms in regularly structured search graphs or search grids. Under certain restrictions, e.g. start and end point restrictions for the path, the shortest path computed corresponds to a recognized word sequence or a generated target language translation. In these algorithms, a shortest-path search  a traveling salesman problem with a0 cities. The visited cities correspond to processed source positions.</Paragraph>
    <Paragraph position="2"> is carried out in one pass over some input along a specific 'direction': in speech recognition the search is timesynchronous, the single-word based search algorithm in (Tillmann et al., 1997) is (source) position-synchronous or left-to-right, the search algorithm in (Niessen et al., 1998) is (target) position-synchronous or bottom-to-top, and the search algorithm in (Tillmann and Ney, 2003) is so-called cardinality-synchronous.</Paragraph>
    <Paragraph position="3"> Taking into account the different word order between source and target language sentences, it becomes less obvious that a SMT search algorithm can be described as a shortest path finding algorithm. But this has been shown by linking decoding to a dynamic-programming solution for the traveling salesman problem. This algorithm due to (Held and Karp, 1962) is a special case of a shortest path finding algorithm (Dreyfus and Law, 1977). The regularly structured search graph for this problem is illustrated in Fig. 1: all paths from the left-most to the right-most vertex correspond to a translation of the in- null put sentence, where each source position is processed exactly once. In this paper, the DP-based search algorithm in (Tillmann and Ney, 2003) is extended in a formal way to handle phrase-based translation. Two versions of a phrase-based decoder for SMT that search slightly different search graphs are presented: a multi-beam decoder reported in the literature and a single-beam decoder with increased translation speed 1. A common analysis of all the search algorithms above in terms of a shortest-path finding algorithm for a directed acyclic graph (dag) is presented. This analysis provides a simple way of analyzing the complexity of DP-based search algorithm.</Paragraph>
    <Paragraph position="4"> Generally, the regular search space can only be fully searched for small search grids under appropriate restrictions, i.e. the monotonicity restrictions in (Tillmann et al., 1997) or the inverted search graph in (Niessen et al., 1998). For larger search spaces as are required for continuous speech recognition (Ney et al., 1992) 2 or phrase-based decoding in SMT, the search space cannot be fully searched: suitably defined lists of path hypothesis are maintained that partially explore the search space. The number of hypotheses depends locally on the number hypotheses whose score is close to the top scoring hypothesis: this set of hypotheses is called the beam.</Paragraph>
    <Paragraph position="5"> The translation model used in this paper is a phrase-based model, where the translation units are so-called blocks: a block is a pair of phrases which are translations of each other. For example, Fig. 2 shows an Arabic-English translation example that uses a0 blocks. During decoding, we view translation as a block segmentation process, where the input sentence is segmented from left to right and the target sentence is generated from bottom to top, one block at a time. In practice, a largely monotone block sequence is generated except for the possibility to swap some neighbor blocks. During decoding, we try to minimize the score a1a3a2a5a4a7a6a9a8a10a12a11 of a block sequence a6a9a8a10 under the restriction that the concatenated source phrases of the blocks a6a14a13 yield a segmentation of the input sentence: null</Paragraph>
    <Paragraph position="7"> a6a9a13a9a11 is a36 -dimensional feature vector with real-valued features and a27 is the corresponding weight vector as described in Section 5. The fact that a given block covers some source interval a37a38a25a39 a23 a38a25a40 is implicit in this notation.</Paragraph>
    <Paragraph position="8"> 1The multi-beam decoder is similar to the decoder presented in (Koehn, 2004) which is a standard decoder used in phrase-based SMT. A multi-beam decoder is also used in (Al-Onaizan et al., 2004) and (Berger et al., 1996).</Paragraph>
    <Paragraph position="9"> 2In that work, there is a distinction between within-word and between-word search, which is not relevant for phrase-based decoding where only exact phrase matches are searched.</Paragraph>
    <Paragraph position="11"> where the Arabic words are romanized. A sequence of a0 blocks is generated.</Paragraph>
    <Paragraph position="12"> This paper is structured as follows: Section 2 introduces the multi-beam and the single-beam DP-based decoders. Section 3 presents an analysis of all the graph-based shortest-path finding algorithm mentioned above: a search algorithm for a directed acyclic graph (dag).</Paragraph>
    <Paragraph position="13"> Section 4 shows an efficient phrasal alignment algorithm that gives an algorithmic justification for learning blocks from word-aligned training. Finally, Section 5 presents an evaluation of the beam-search decoders on an Arabic-English decoding task.</Paragraph>
  </Section>
class="xml-element"></Paper>