<?xml version="1.0" standalone="yes"?>
<Paper uid="P95-1035">
  <Title>An Efficient Generation Algorithm for Lexicalist MT</Title>
  <Section position="3" start_page="261" end_page="262" type="metho">
    <SectionTitle>
2 A Greedy Incremental Generation
Algorithm
</SectionTitle>
    <Paragraph position="0"> We begin by describing the fundamentals of a greedy incremental generation algorithm. The cruciM data structure that it employs is the TNCB. We give some definitions, state some key assumptions about suitable TNCBs for generation, and then describe the algorithm itself.</Paragraph>
    <Section position="1" start_page="261" end_page="262" type="sub_section">
      <SectionTitle>
2.1 TNCBs
</SectionTitle>
      <Paragraph position="0"> We assume a sign-based grammar with binary rules, each of which may be used to combine two signs by unifying them with the daughter categories and returning the mother. Combination is the commutative equivalent of rule application; the linear ordering of the daughters that leads to successful rule application determines the orthography of the mother.</Paragraph>
      <Paragraph position="1"> Whitelock's Shake-and-Bake generation algorithm attempts to arrange the bag of target signs until a grammatical ordering (an ordering which allows all of the signs to combine to yield a single sign) is found. However, the target derivation information itself is not used to assist the algorithm. Even in (Beaven, 1992a), the derivation information is used simply to cache previous results to avoid exact re-computation at a later stage, not to improve on previous guesses. The reason why we believe such improvement is possible is that, given adequate information from the previous stages, two target signs cannot combine by accident; they must do so because the underlying semantics within the signs licenses it.</Paragraph>
      <Paragraph position="2"> If the linguistic data that two signs contain allows them to combine, it is because they are providing a semantics which might later become more specified. For example, consider the bag of signs that have been derived through the Shake-and-Bake process which represent the phrase:  (1) The big brown dog Now, since the determiner and adjectives all modify the same noun, most grammars will allow us to construct the phrases: (2) The dog (3) The big dog (4) The brown dog  as well as the 'correct' one. Generation will fail if all signs in the bag are not eventually incorporated in tile final result, but in the naive algorithm, the intervening computation may be intractable.</Paragraph>
      <Paragraph position="3"> In the algorithm presented here, we start from observation that the phrases (2) to (4) are not incorrect semantically; they are simply under-specifications of (1). We take advantage of this by recording the constituents that have combined within the TNCB, which is designed to allow further constituents to be incorporated with minimal recomputation.</Paragraph>
      <Paragraph position="4"> A TNCB is composed of a sign, and a history of how it was derived from its children. The structure is essentially a binary derivation tree whose children are unordered. Concretely, it is either NIL, or a triple:</Paragraph>
      <Paragraph position="6"> The second and third items of the TNCB triple are the child TNCBs. The value of a TNCB is the sign that is formed from the combination of its children, or INCONSISTENT, representing the fact that they cannot grammatically combine, or UN-DETERMINED, i.e. it has not yet been established whether the signs combine.</Paragraph>
      <Paragraph position="7"> Undetermined TNCBs are commutative, e.g. they do not distinguish between the structures shown in  In section 3 we will see that this property is important when starting up the generation process. Let us introduce some terminology.</Paragraph>
      <Paragraph position="8">  A TNCB is * well-formed iff its value is a sign, * ill-formed iff its value is INCONSISTENT, * undetermined (and its value is UNDETER-MINED) iff it has not been demonstrated whether it is well-formed or ill-formed.</Paragraph>
      <Paragraph position="9"> * maximal iff it is well-formed and its parent (if it has one) is ill-formed. In other words, a maximal TNCB is a largest well-formed component of a TNCB.</Paragraph>
      <Paragraph position="10">  Since TNCBs are tree-like structures, if a TNCB is undetermined or ill-formed then so are all of its ancestors (the TNCBs that contain it). We define five operations on a TNCB. The first three are used to define the fourth transformation (move) which improves ill-formed TNCBs. The fifth is used to establish the well-formedness of undetermined nodes. In the diagrams, we use a cross to represent ill-formed nodes and a black circle to represent undetermined ones.</Paragraph>
      <Paragraph position="11"> Deletion: A maximal TNCB can be deleted from its current position. The structure above it must be adjusted in order to maintain binary branching. In figure 2, we see that when node  4 is deleted, so is its parent node 3. The new node 6, representing the combination of 2 and 5, is marked undetermined.</Paragraph>
      <Paragraph position="12">  Conjunction: A maximal TNCB can be conjoined with another maximal TNCB if they may be combined by rule. In figure 3, it can be seen how the maximal TNCB composed of nodes 1, 2, and 3 is conjoined with the maximal TNCB composed of nodes 4, 5 and 6 giving the TNCB made up of nodes 1 to 7. The new node, 7, is</Paragraph>
      <Paragraph position="14"> Figure 3:1 is conjoined with 4 giving 7 Adjunction: A maximal TNCB can be inserted inside a maximal TNCB, i.e. conjoined with a non-maximal TNCB, where the combination is licensed by rule. In figure 4, the TNCB composed of nodes 1, 2, and 3 is inserted inside the TNCB composed of nodes 4, 5 and 6.</Paragraph>
      <Paragraph position="15"> All nodes (only 8 in figure 4) which dominate the node corresponding to the new combination (node 7) must be marked undetermined -- such nodes are said to be disrupted.</Paragraph>
      <Paragraph position="16">  Movement: This is a combination of a deletion with a subsequent conjunction or adjunction. In figure 5, we illustrate a move via conjunction. In the left-hand figure, we assume we wish to move the maximal TNCB 4 next to the maximal TNCB 7. This first involves deleting TNCB 4 (noting it), and raising node 3 to replace node 2. We then introduce node 8 above node 7, and make both nodes 7 and 4 its children. Note that during deletion, we remove a surplus node (node 2 in this case) and during conjunction or adjunction we introduce a new one (node 8 in this case) thus maintaining the same number of nodes in the tree.</Paragraph>
    </Section>
  </Section>
  <Section position="4" start_page="262" end_page="264" type="metho">
    <SectionTitle>
9 /L
</SectionTitle>
    <Paragraph position="0"> Evaluation: After a movement, the TNCB is undetermined as demonstrated in figure 5.</Paragraph>
    <Paragraph position="1"> The signs of the affected parts must be recalculated by combining the recursively evaluated child TNCBs.</Paragraph>
    <Section position="1" start_page="262" end_page="263" type="sub_section">
      <SectionTitle>
2.2 Suitable Grammars
</SectionTitle>
      <Paragraph position="0"> The Shake-and-Bake system of (Whitelock, 1992) employs a bag generation algorithm because it is assumed that the input to the generator is no more than a collection of instantiated signs. Full-scale bag generation is not necessary because sufficient information can be transferred from the source language to severely constrain the subsequent search during generation.</Paragraph>
      <Paragraph position="1"> The two properties required of TNCBs (and hence the target grammars with instantiated lexicM signs)  are: 1. Precedence Monotonicity. The order of the  orthographies of two combining signs in the orthography of the result must be determinate -it must not depend on any subsequent combination that the result may undergo. This constraint says that if one constituent fails to combine with another, no permutation of the elements making up either would render the combination possible. This allows bottom-up evaluation to occur in linear time. In practice, this restriction requires that sufficiently rich information be transferred from the previous translation stages to ensure that sign combination is deterministic.</Paragraph>
      <Paragraph position="2">  2. Dominance Monotonicity. If a maximal TNCB is adjoined at the highest possible place inside another TNCB, the result will be well-formed after it is re-evaluated. Adjunction is  only attempted if conjunction fails (in fact conjunction is merely a special case of adjunction in which no nodes are disrupted); an adjunction which disrupts i nodes is attempted before one which disrupts i + 1 nodes. Dominance monotonicity merely requires all nodes that are disrupted under this top-down control regime to be well-formed when re-evaluated. We will see that this will ensure the termination of the generation algorithm within n- 1 steps, where n is the number of lexical signs input to the process. We are currently investigating the mathematical characterisation of grammars and instantiated signs that obey these constraints. So far, we have not found these restrictions particularly problematic.</Paragraph>
    </Section>
    <Section position="2" start_page="263" end_page="264" type="sub_section">
      <SectionTitle>
2.3 The Generation Algorithm
</SectionTitle>
      <Paragraph position="0"> The generator cycles through two phases: a test phase and a rewrite phase. Imagine a bag of signs, corresponding to &amp;quot;the big brown dog barked&amp;quot;, has been passed to the generation phase. The first step in the generation process is to convert it into some arbitrary TNCB structure, say the one in figure 6.</Paragraph>
      <Paragraph position="1"> In order to verify whether this structure is valid, we evaluate the TNCB. This is the test phase. If the TNCB evaluates successfully, the orthography of its value is the desired result. If not, we enter the rewrite phase.</Paragraph>
      <Paragraph position="2"> If we were continuing in the spirit of the original Shake-and-Bake generation process, we would now form some arbitrary mutation of the TNCB and retest, repeating this test-rewrite cycle until we either found a well-formed TNCB or failed. However, this would also be intractable due to the undirectedness of the search through the vast number of possibilities. Given the added derivation information contained within TNCBs and the properties mentioned above, we can direct this search by incrementally improving on previously evaluated results.</Paragraph>
      <Paragraph position="3"> We enter the rewrite phase, then, with an ill-formed TNCB. Each move operation must improve</Paragraph>
      <Paragraph position="5"> it. Let us see why this is so.</Paragraph>
      <Paragraph position="6"> The move operation maintains the same number of nodes in the tree. The deletion of a maximal TNCB removes two ill-formed nodes (figure 2). At the deletion site, a new undetermined node is created, which may or may not be ill-formed. At the destination site of the movement (whether conjunction or adjunction), a new well-formed node is created. null The ancestors of the new well-formed node will be at least as well-formed as they were prior to the movement. We can verify this by case:  1. When two maximal TNCBs are conjoined, nodes dominating the new node, which were previously ill-formed, become undetermined.</Paragraph>
      <Paragraph position="7"> When re-evaluated, they may remain ill-formed or some may now become well-formed.</Paragraph>
      <Paragraph position="8"> 2. When we adjoin a maximal TNCB within an null other TNCB, nodes dominating the new well-formed node are disrupted. By dominance monotonicity, all nodes which were disrupted by the adjunction must become well-formed after re-evaluation. And nodes dominating the maximal disrupted node, which were previously ill-formed, may become well-formed after reevaluation. null We thus see that rewriting and re-evaluating must improve the TNCB.</Paragraph>
      <Paragraph position="9"> Let us further consider the contrived worst-case starting point provided in figure 6. After the test phase, we discover that every single interior node is ill-formed. We then scan the TNCB, say top-down from left to right, looking for a maximal TNCB to move. In this case, the first move will be PAST to bark, by conjunction (figure 7).</Paragraph>
      <Paragraph position="10"> Once again, the test phase fails to provide a well-formed TNCB, so we repeat the rewrite phase, this time finding dog to conjoin with the (figure 8 shows the state just after the second pass through the test phase).</Paragraph>
      <Paragraph position="11"> After further testing, we again re-enter the rewrite phase and this time note that brown can be inserted in the maximal TNCB the dog barked adjoined with dog (figure 9). Note how, after combining dog and the, the parent sign reflects the correct orthography  After finding that big may not be conjoined with the brown dog, we try to adjoin it within the latter. Since it will combine with brown dog, no adjunction to a lower TNCB is attempted.</Paragraph>
      <Paragraph position="12"> The final result is the TNCB in figure 11, whose orthography is &amp;quot;the big brown dog barked&amp;quot;. We thus see that during generation, we formed a basic constituent, the dog, and incrementally refined it by adjoining the modifiers in place. At the heart of this approach is that, once well-formed, constituents can only grow; they can never be dismantled.</Paragraph>
      <Paragraph position="13"> Even if generation ultimately fails, maximal well-formed fragments will have been built; the latter may be presented to the user, allowing graceful degradation of output quality.</Paragraph>
    </Section>
  </Section>
  <Section position="5" start_page="264" end_page="265" type="metho">
    <SectionTitle>
3 Initialising the Generator
</SectionTitle>
    <Paragraph position="0"> Considering the algorithm described above, we note that the number of rewrites necessary to repair the initial guess is no more than the number of ill-formed TNCBs. This can never exceed the number of interior nodes of the TNCB formed from n lexical signs (i.e. n-2). Consequently, the better formed the initial TNCB used by the generator, the fewer the number of rewrites required to complete generation. In the last section, we deliberately illustrated an initial guess which was as bad as possible. In this section, we consider a heuristic for producing a motivated guess for the initial TNCB.</Paragraph>
    <Paragraph position="1"> Consider the TNCBs in figure 1. If we interpret the S, O and V as Subject, Object and Verb we can observe an equivalence between the structures with the bracketings: (S (V O)), (S (O V)), ((V O) S), and ((O V) S). The implication of this equivalence is that if, say, we are translating into a (S (V O)) language from a head-finM language and have isomorphic dominance structures between the source and target parses, then simply mirroring the source parse structure in the initial target TNCB will provide a correct initiM guess. For example, the English sentence (5): (5) the book is red  has a corresponding Japanese equivalent (6): (6) ((hon wa) (akai desu)) ((book TOP) (red is)) If we mirror the Japanese bracketing structure in English to form the initial TNCB, we obtain: ((book the) (red is)). This will produce the correct answer in the test phase of generation without the need to rewrite at all.</Paragraph>
    <Paragraph position="2"> Even if there is not an exact isomorphism between the source and target commutative bracketings, the first guess is still reasonable as long as the majority of child commutative bracketings in the target language are isomorphic with their equivalents in the source language. Consider the French sentence: (7) ((le ((grandchien) brun)) aboya) (8) ((the ((big dog) brown)) barked) The TNCB implied by the bracketing in (8) is equivalent to that in figure 10 and requires just one rewrite in order to make it well-formed. We thus see how the TNCBs can mirror the dominance information in the source language parse in order to furnish the generator with a good initial guess. On the other hand, no matter how the SL and TL structures differ, the algorithm will still operate correctly with polynomial complexity. Structural transfer can be incorporated to improve the efficiency of generation, but it is never necessary for correctness or even tractability.</Paragraph>
  </Section>
  <Section position="6" start_page="265" end_page="265" type="metho">
    <SectionTitle>
4 The Complexity of the Generator
</SectionTitle>
    <Paragraph position="0"> The theoretical complexity of the generator is O (n4), where n is the size of the input. We give an informal argument for this. The complexity of the test phase is the number of evaluations that have to be made.</Paragraph>
    <Paragraph position="1"> Each node must be tested no more than twice in the worst case (due to precedence monotonicity), as one might have to try to combine its children in either direction according to the grammar rules. There are always exactly n - 1 non-leaf nodes, so the complexity of the test phase is O(n). The complexity of the rewrite phase is that of locating the two TNCBs to be combined. In the worst case, we can imagine picking an arbitrary child TNCB (O(n)) and then trying to find another one with which it combines (O(n)). The complexity of this phase is therefore the product of the picking and combining complexities, i.e. O(n2). The combined complexity of the test-rewrite cycle is thus O(n3). Now, in section 3, we argued that no more than n - 1 rewrites would ever be necessary, thus the overall complexity of generation (even when no solution is found) is O(n4).</Paragraph>
    <Paragraph position="2"> Average case complexity is dependent on the quality of the first guess, how rapidly the TNCB structure is actually improved, and to what extent the TNCB must be re-evaluated after rewriting. In the SLEMaT system (Poznarlski et al., 1993), we have tried to form a good initial guess by mirroring the source structure in the target TNCB, and allowing some local structural modifications in the bilingual equivalences.</Paragraph>
    <Paragraph position="3"> Structural transfer operations only affect the efficiency and not the functionality of generation.</Paragraph>
    <Paragraph position="4"> Transfer specifications may be incrementally refined and empirically tested for efficiency. Since complete specification of transfer operations is not required for correct generation of grammatical target text, the version of Shake-and-Bake translation presented here maintains its advantage over traditional transfer models, in this respect.</Paragraph>
    <Paragraph position="5"> The monotonicity constraints, on the other hand, might constitute a dilution of the Shake-and-Bake ideal of independent grammars. For instance, precedence monotonicity requires that the status of a clause (strictly, its lexical head) as main or subordinate has to be transferred into German. It is not that the transfer of information per se compromises the ideal -- such information must often appear in transfer entries to avoid grammatical but incorrect translation (e.g. a great man translated as un homme grand). The problem is justifying the main/subordinate distinction in every language that we might wish to translate into German. This distinction can be justified monolingually for the other languages that we treat (English, French, and Japanese). Whether the constraints will ultimately require monolingual grammars to be enriched with entirely unmotivated features will only become clear as translation coverage is extended and new language pairs are added.</Paragraph>
  </Section>
class="xml-element"></Paper>