File Information
File: 05-lr/acl_arc_1_sum/cleansed_text/xml_by_section/metho/02/w02-1039_metho.xml
Size: 19,760 bytes
Last Modified: 2025-10-06 14:08:02
<?xml version="1.0" standalone="yes"?> <Paper uid="W02-1039"> <Title>Phrasal Cohesion and Statistical Machine Translation</Title> <Section position="3" start_page="0" end_page="0" type="metho"> <SectionTitle> 2 Alignments, Spans and Crossings </SectionTitle> <Paragraph position="0"> An alignment is a mapping between the words in a string in one language and the translations of those words in a string in another language. Given an English string, a0a2a1a4a3a6a5a7a9a8a10a3 a7 a3a12a11a14a13a15a13a15a13a16a3 a5 , and a French string, a17 a1a19a18a21a20a7 a8a19a18 a7 a18 a11 a13a15a13a15a13a22a18 a20 , an alignment a can be represented by a23 a1a2a24a25a5a7a26a8a27a24 a7 a24 a11 a13a15a13a15a13a28a24 a5 . Each a24a30a29 is a set of indices into a0 where a31a33a32 a24a25a29a35a34a12a36a38a37 a31 a37a9a39a40a34a42a41a43a37 a37a46a45 indicates that word a31 in the French sentence is aligned with word a44 in the English sentence. a24 a29 a1a48a47 indicates that English word a44 has no corresponding French word.</Paragraph> <Paragraph position="1"> Given an alignment a23 and an English phrase covering words a3a6a29a25a13a15a13a15a13a22a3a28a49 , the span is a pair where the first element is a50a43a51a53a52a55a54 a24 a29a57a56 a13a15a13a15a13 a56 a24 a49a59a58 and the second element is a50a43a60a62a61a63a54 a24a64a29 a56 a13a15a13a15a13 a56 a24a62a49 a58 . Thus, the span includes all words between the two extrema of the alignment, whether or not they too are part of the translation. If phrases cohere perfectly across languages, the span of one phrase will never overlap the span of another.</Paragraph> <Paragraph position="2"> If two spans do overlap, we call this a crossing.</Paragraph> <Paragraph position="3"> Figure 1 shows an example of an English parse along with the alignment between the English and French words (shown with dotted lines). The English word &quot;not&quot; is aligned to the two French words &quot;ne&quot; and &quot;pas&quot; and thus has a span of [1,3]. The main English verb &quot;change&quot; is aligned to the French &quot;modifie&quot; and has a span of [2,2]. The two spans overlap and thus there is a crossing. This definition is asymmetric (i.e. what is a crossing when moving from English to French is not guaranteed to be a crossing when moving from French to English).</Paragraph> <Paragraph position="4"> However, we only pursue translation direction since that is the one for which we have parsed data.</Paragraph> </Section> <Section position="4" start_page="0" end_page="7" type="metho"> <SectionTitle> 3 Experiments </SectionTitle> <Paragraph position="0"/> <Section position="1" start_page="0" end_page="7" type="sub_section"> <SectionTitle> 3.1 Data </SectionTitle> <Paragraph position="0"> To calculate spans, we need aligned pairs of English and French sentences along with parses for the English sentences. Our aligned data comes from a corpus described in (Och and Ney, 2000) which contains 500 sentence pairs randomly selected from the Canadian Hansard corpus and manually aligned.</Paragraph> <Paragraph position="1"> The alignments are of two types: sure (S) and possible (P). S alignments are those which are unamsituation .[ne not vraiment la0 1 2 3 4 5 6 biguous while P alignments are those which are less certain. P alignments often appear when a phrase in one language translates as a unit into a phrase in the other language (e.g. idioms, free translations, missing function words) but can also be the result of genuine ambiguity. When two annotators disagree, the union of the P alignments produced by each annotator is recorded as the P alignment in the corpus. When an S alignment exists, there will always also exist a P alignment such that P a65 S. The English sentences were parsed using a state-of-the-art statistical parser (Charniak, 2000) trained on the University of Pennsylvania Treebank (Marcus et al., 1993).</Paragraph> </Section> <Section position="2" start_page="7" end_page="7" type="sub_section"> <SectionTitle> 3.2 Phrasal Translation Filtering </SectionTitle> <Paragraph position="0"/> <Paragraph position="2"> tions, the number of crossings when P alignments are used will be artificially inflated. For example, in Figure 2 note that every pair of English and French words under the verb phrase is aligned. This will generate five crossings, one each between the pairs VBP-PP, IN-NPa1 , NPa11 -PP, NN-DT, and IN-NPa7 .</Paragraph> <Paragraph position="3"> However, what is really happening is that the whole verb phrase is first being moved without crossing anything else and then being translated as a unit. For our purposes we want to count this example as producing zero crossings. To accomplish this, we defined a simple heuristic to detect phrasal translations so we can filter them if desired.</Paragraph> </Section> <Section position="3" start_page="7" end_page="7" type="sub_section"> <SectionTitle> 3.3 Calculating Crossings </SectionTitle> <Paragraph position="0"> After calculating the French spans from the English parses and alignment information, we counted crossings for all pairs of child constituents in each constituent in the sentence, maintaining separate counts for those involving the head constituent of the phrase and for crossings involving modifiers only. We did this while varying conditions along two axes: alignment type and phrasal translation filtering. Recalling the two different types of alignments, S and P, we examined three different conditions: S alignments only, P alignments only, or S alignments where present falling back to P alignments (Sa0 P). For each of these conditions, we counted crossings both with and without using the phrasal translation filter.</Paragraph> <Paragraph position="1"> For a given alignment type a2 a32a4a3 S, S a0 P,Pa5 , let</Paragraph> <Paragraph position="3"> a11 cross each other and a41 otherwise. Let a11 a7 a54a10a9 a7 a56 a9 a11 a58 a8a19a36 if the phrasal translation filter is turned off. If the filter is on,</Paragraph> <Paragraph position="5"> Then, for a given phrase a9 with head constituent a5 and for a particular alignment type a2 , the number of head crossings a6a22a21 a7 and modifier crossings a6 a20a7 can be calculated recursively:</Paragraph> <Paragraph position="7"/> </Section> </Section> <Section position="5" start_page="7" end_page="7" type="metho"> <SectionTitle> 4 Results </SectionTitle> <Paragraph position="0"/> <Section position="1" start_page="7" end_page="7" type="sub_section"> <SectionTitle> 4.1 Average Crossings </SectionTitle> <Paragraph position="0"> Table 1 shows the average number of crossings per sentence. The table is split into two sections, one for results when the phrasal filter was used and one for when it was not. &quot;Alignment Type&quot; refers to whether we used S, P or Sa0 P as the alignment data. The &quot;Head Crossings&quot; line shows the results when comparing the span of the head constituent of a phrase with the spans of its modifier constituents, and &quot;Modifier Crossings&quot; refers to the case where we compare the spans of pairs of modifiers. The &quot;Phrasal Translations&quot; line shows the average number of phrasal translations detected per sentence.</Paragraph> <Paragraph position="1"> For S alignments, the results are quite promising, with an average of only 0.236 head crossings per sentence and an even smaller average for modifier crossings (0.056). However, these results are overly optimistic since often many words in a sentence will not have an S alignment at all, such as &quot;coming&quot;, &quot;in&quot;, and &quot;before&quot; in following example: le rapport complet sera de ici le automne prochaind'epos'e the full report will be coming in before the fall When we use P alignments for these unaligned words (the Sa0 P case), we get a more meaningful result. Both types of crossings are much more frequent (4.790 for heads and 0.88 for modifiers) and phrasal translation filtering has a much larger effect (reducing head average to 2.772 and modifier average to 0.516). Phrasal translations account for almost half of all crossings, on average. This effect is even more pronounced in the case where we use P alignments only. This reinforces the importance of phrasal translation in the development of any translation system.</Paragraph> <Paragraph position="2"> Even after filtering, the number of crossings in the Sa0 P case is quite large. This is discouraging, however there are reasons why this result should be looked on as more of an upper bound than anything precise. For one thing, there are cases of phrasal translation which our heuristic fails to recognize, an example of which is shown in Figure 3. The alignment of &quot;explorer&quot; with &quot;this&quot; and &quot;matter&quot; seems to indicate that the intention of the annotator was to align the phrase &quot;work this matter out&quot;, as a unit, to &quot;de explorer la question&quot;. However, possibly due to an error during the coding of the alignment, &quot;work&quot; and &quot;out&quot; align with &quot;de&quot; (indicated by the solid lines) while &quot;this&quot; and &quot;matter&quot; do not. This causes the phrasal translation heuristic to fail resulting in a crossing where there should be none.</Paragraph> <Paragraph position="3"> Also, due to the annotation guidelines, P alignments are not as consistent as would be ideal. Recall that in cases of annotator disagreement, the P alignment is taken to be the union of the P alignments of both annotators. Thus, it is possible for the P alignment to contain two mutually conflicting alignments. These composite alignments will likely generate crossings even where the alignments of each individual annotator would not. While reflecting genuine ambiguity, an SMT system would likely pursue only one of the alternatives and only a portion of the crossings would come into play.</Paragraph> </Section> <Section position="2" start_page="7" end_page="7" type="sub_section"> <SectionTitle> 4.2 Percentage Crossings </SectionTitle> <Paragraph position="0"> Our results show a significantly larger number of head crossings than modifier crossings. One possibility is that this is due to most phrases having a head and modifier pair to test, while many do not have multiple modifiers and therefore there are fewer opportunities for modifier crossings. Thus, it is informative to examine how many potential crossings actually turn out to be crossings. Table 2 provides this result in the form of the percentage of crossing tests which result in detection of a crossing.</Paragraph> <Paragraph position="1"> To calculate this, we kept totals for the number of head (a0 a21a7 ) and modifier (a0 a20a7 ) crossing tests performed as well as the number of phrasal translations detected (a0a2a1 a21a7 ). Note that when the phrasal translation filter is turned on, these totals differ for each of the different alignment types (S, Sa0 P, and P).</Paragraph> <Paragraph position="3"> The percentages are calculated after summing over all sentences a11 in the corpus:</Paragraph> <Paragraph position="5"> There are still many more crossings in the Sa0 P and P alignments than in the S alignments. The S alignment has 1.58% head crossings while the Sa0 P and P alignments have 32.16% and 35.47% respectively, with similar relative percentages for modifier crossings. Also as before, half to two-thirds of crossings in the Sa0 P and P alignments are due to phrasal translations. More interestingly, we see that modifier crossings remain significantly less prevalent than head crossings (e.g. 14.45% vs. 32.16% for the Sa0 P case) and that this is true uniformly across all parameter settings. This indicates that heads are more intimately involved with their modifiers than modifiers are with each other and therefore are more likely to be involved in semi-phrasal constructions.</Paragraph> </Section> </Section> <Section position="6" start_page="7" end_page="7" type="metho"> <SectionTitle> 5 Analysis of Causes </SectionTitle> <Paragraph position="0"> Since it is clear that crossings are too prevalent to ignore, it is informative to try to understand exactly what constructions give rise to them. To that end, we examined by hand all of the head crossings produced using the S alignments with phrasal filtering. Table 3 shows the results of this analysis.</Paragraph> <Paragraph position="1"> The first thing to note is that by far most of the crossings do not reflect lack of phrasal cohesion between the two languages. Instead, they are caused either by errors in the syntactic analysis or the fact that translation as done by humans is a much richer process than just replication of the source sentence in another language. Sentences are reworded, clauses are reordered, and sometimes human translators even make mistakes.</Paragraph> <Paragraph position="2"> Errors in syntactic analysis consist mostly of attachment errors. Rewording and reordering accounted for a large number of crossings as well. In most of the cases of rewording (see Figure 4) or reaura de les effets destructifsplus que positifsen fait , elle ordering (see Figure 5) a more &quot;parallel&quot; translation would also be valid. Thus, while it would be difficult for a statistical model to learn from these examples, there is nothing to preclude production of a valid translation from a system using phrasal movement in the reordering phase. The rewording and reordering examples were so varied that we were unable to find any regularities which might be exploited by a translation model.</Paragraph> <Paragraph position="3"> Among the cases which do result from language differences, the most common is the &quot;ne . . . pas&quot; construction (e.g. Figure 1). Fifteen percent of the 86 total crossings are due to this construction. Because &quot;ne . . . pas&quot; wraps around the verb, it will always result in a crossing. However, the types of syntactic structures (categorized as context-free grammar rules) which are present in cases of negation are rather restricted. Of the 47 total distinct syntactic structures which resulted in crossings, only three of them involved negation. In addition, the crossings associated with these particular structures were unambiguously caused by negation (i.e. for each structure, only negation-related crossings were present). Next most common is the case where the English contains a modal verb which is aligned with the main verb in the French. In the example in Figure 6, &quot;will be&quot; is aligned to &quot;sera&quot; (indicated by the solid lines) and because of the constituent structure of the English parse there is a crossing. As with negation, this type of crossing is quite regular, resulting uniquely from only two different syntactic structures.</Paragraph> <Paragraph position="4"> le rapport complet sera de ici le automne prochaind'epos'e Adverbs are a third common cause, as they typically follow the verb in French while preceding it in English. Figure 7 shows an example where the span of &quot;simplement&quot; overlaps with the span of the verb phrase beginning with &quot;tells&quot; (indicated by the solid lines). Unlike negation and modals, this case is far less regular. It arises from six different syntactic constructions and two of those constructions are implicated in other types of crossings as well.</Paragraph> <Paragraph position="5"> les bontout simplement gens ce est pour euxle gouvernement dit qui`a</Paragraph> </Section> <Section position="7" start_page="7" end_page="7" type="metho"> <SectionTitle> 6 Further Experiments </SectionTitle> <Paragraph position="0"/> <Section position="1" start_page="7" end_page="7" type="sub_section"> <SectionTitle> 6.1 Flattening Verb Phrases </SectionTitle> <Paragraph position="0"> Many of the causes listed above are related to verb phrases. In particular, some of the adverb-related crossings (e.g. Figure 1) and all of the modal-related crossings (e.g. Figure 6) are artifacts of the nested verb phrase structure of our parser. This nesting usually does not provide any extra information beyond what could be gleaned from word order. Therefore, we surmised that flattening verb phrases would eliminate some types of crossings without reducing the utility of the parse.</Paragraph> <Paragraph position="1"> The flattening operation consists of identifying all nested verb phrases and splicing the children of the nested phrase into the parent phrase in its place. This procedure is applied recursively until there are no nested verb phrases. An example is shown in Figure 8. Crossings can be calculated as before.</Paragraph> <Paragraph position="2"> Flattening reduces the number of potential head crossings while increasing the number of potential modifier crossings. Therefore, we would expect to see a comparable change to the number of crossings measured, and this is exactly what we find, as shown in Tables 4 and 5. For example, for Sa0 P alignments, the average number of head crossings decreases from 2.772 to 2.252, while the average number of modifier crossings increases from 0.516 to 0.86. We see similar behavior when we look at the percentage of crossings per chance (Tables 6 and 7).</Paragraph> <Paragraph position="3"> For the same alignment type, the percentage of head crossings decreases from 18.61% to 15.12%, while the percentage of modifier crossings increases from 8.47% to 10.59%. One thing to note, however, is that the total number of crossings of both types detected in the corpus decreases as compared to the baseline, and thus the benefits to head crossings outweigh the detriments to modifier crossings.</Paragraph> </Section> <Section position="2" start_page="7" end_page="7" type="sub_section"> <SectionTitle> 6.2 Dependencies </SectionTitle> <Paragraph position="0"> Our intuitions about the cohesion of syntactic structures follow from the notion that translation, as a meaning-preserving operation, preserves the dependencies between words, and that syntactic structures encode these dependencies. Therefore, dependency structures should cohere as well as, or better than, their corresponding syntactic structures. To examine the validity of this, we extracted dependency structures from the parse trees (with flattened verb phrases) and calculated crossings for them. Figure 9 shows a parse tree and its corresponding dependency structure.</Paragraph> <Paragraph position="1"> The procedure for counting modifier crossings in a dependency structure is identical to the procedure for parse trees. For head crossings, the only difference is that rather than comparing spans of two siblings, we compare the spans of a child and its parent. bewill before thecoming fall the Again focusing on the Sa0 P alignment case, we see that the average number of head crossings (see Table 4) continues to decrease compared to the previous case (from 2.252 to 1.88), and that the average number of modifier crossings (see Table 5) continues to increase (from 0.86 to 1.498). This time, however, the percentages for both types of crossings (see Tables 6 and 7) decrease relative to the case of flattened verb phrases (from 15.12% to 12.62% for heads and from 10.59% to 9.22% for modifiers).</Paragraph> <Paragraph position="2"> The percentage of modifier crossings is still higher than in the base case (9.22% vs. 8.47%). Overall, however, the dependency representation has the best cohesion properties.</Paragraph> </Section> </Section> class="xml-element"></Paper>