Chart-Based Transfer Rule Application in Machine Translation 
Adam Meyers 
New York University 
meyers@cs.nyu.edu 
Michiko Kosaka 
Monlnouth University 
kosaka@monmouth.edu 
Ralph GrishInan 
New York University 
grishman@cs.nyu.edu 
Abstract 
35"ansfer-based Machine Translation systems re- 
quire a procedure for choosing the set; of transfer 
rules for generating a target  transla- 
tion from a given source  sentence. In 
an MT system with many comI)eting transfer 
rules, choosing t;he best, set of transfer rules for 
translation may involve the evaluation of an ex- 
plosive number of competing wets. We propose a 
sohltion t;o this problem l)ased on current best- 
first chart parsing algorithms. 
1 Introduction 
ri~'ansfer-based Machine 'Kanslation systenls re- 
quire a procedure for choosing the set of trans- 
tier rules for generating a target  I;rans- 
lation from a given source  sentence. 
This procedure is trivial tbr a system if, given 
a (:ontext, one transtb.r rule. can l)e selected un- 
~mfl)iguously. O|;herwise, choosing the besl; set; 
of transfer rules may involve the. evaluation of 
mmmrous competing sets. In fact, the number 
of l)ossible transfer rule combinations increas- 
es exponentially with the length of the source, 
 sentence,. This situation mirrors the 
t)roblem of choosing productions in a nondeter- 
ministic parser, in this paI)er, we descril)e a 
system for choosing transfer rules, based on s- 
tatistical chart parsing (Bol)row, 1990; Chitrao 
and Grishman, 1990; Caraballo and Charniak, 
1997; Charniak et al., 1998). 
In our Machine %'anslation system, transfer 
rules are generated automatically from parsed 
parallel text along the lines of (Matsulnoto el; 
al,, 1993; Meyers et al., 1996; Meyers et al., 
1998b). Our system tends to acquire a large 
nmnber of transt~r rules, due lnainly to 3,1terna- 
tive ways of translating the same sequences of 
words, non-literal translations in parallel text 
and parsing e, rrors. It is therefore crucial that 
our system choose the best set of rules efficient- 
ly. While the technique discussed he.re obviously 
applies to similar such systems, it could also ap- 
ply to hand-coded systems in which each word 
or group of words is related to more than one 
transfer rule. D)r example, both Multra (Hein, 
1996) and the Eurotra system described in (Way 
el; al., 1997) require components for deciding 
which combination of transtbr rules to use. The 
proi).osed technique may 1)e used with syst;ems 
like these, t)rovided that all transfer rules are as- 
signed initial scores rating thcqr at)propriateness 
for translation. These al)t)rol)riateness ratings 
couhl be dependent or independent of context. 
2 Previous Work 
The MT literature deserib(;s several techniques 
tbr deriving the appropriate translation. Statis- 
tical systems l;hal; do not incorporate linguistic 
analysis (Brown el: al., 1993) typically choose 
the most likely translation based on a statis- 
tical mode.l, i.e.., translation probability deter- 
mines the translation. (Hein, 1996) reports a set; 
of (hand-coded) fea|;llre structure based prefi~r- 
ence rules to choose among alternatives in Mu\]- 
tra. There is some discussion about adding 
some transtbr rules automatically acquired flom 
corpora to Multra? Assuming that they over- 
generate rules (as we did), a system like the one 
we propose should 1)e beneficial. In (Way et al., 
1997), many ditDrent criteria are used to dloose 
trmlsi~;r rules to execute including: pretbrmlces 
for specific rules over general ones, and comt)lex 
rule nol, ation that insures that tb.w rules can 21)- 
ply to the same set, of words. 
The Pangloss Mark III system (Nirenburg 
~This translatioll procedm'e would probably comple- 
menI~ not; replace exist, ing procedures in these systelns. 
2http ://stp. ling. uu. se/~corpora/plug/report s / 
ansk_last/ is a report on this 1)reject; for Multra. 
537 
and Frederking, 1995) uses a chart-walk algo- 
rithm to combine the results of three MT en- 
gines: an example-based engine, a knowledge- 
based engine, and a lexical-transfer engine. 
Each engine contributes its best edges and tile 
chart-walk algorithm uses dynamic program- 
ruing to find the combination of edges with the 
best overall score that covers the input string. 
Scores of edges are normalized so that the scores 
fi'om the different engines are comparable and 
weighted to favor engines which tend to produce 
better results. Pangloss's algorithm combines 
whole MT systems. In contrast, our algorith- 
m combines output of individual transfer rules 
within a single MT system. Also, we use a best- 
first search that incorporates a probabilistic- 
based figure of merit, whereas Pangloss uses an 
empirically based weighting scheme and what 
appears to be a top-down search. 
Best-first probabilistic chart parsers (Bo- 
brow, 1990; Chitrao and Grishman, 1990; Cara- 
ballo and Charniak, 1997; Charniak et al., 1998) 
strive to find the best parse, without exhaus- 
tively trying all possible productions. A proba- 
bilistic figure of merit (Caraballo and Charniak, 
1997; Charniak et al., 1998) is devised for rank- 
ing edges. The highest ranking edges are pur- 
sued first and the parser halts after it produces 
a complete parse. We propose an algorithm for 
choosing and applying transthr rules based on 
probability. Each final translation is derived 
from a specific set of transfer rules. If the pro- 
cedure immediately selected these transfer rules 
and applied them in tile correct order, we would 
arrive at tile final translation while creating the 
minimum number of edges. Our procedure uses 
about 4 tinms this minimum number of edges. 
With respect to chart parsing, (Charniak et al., 
1998) report that their parser can achieve good 
results while producing about three times tile 
mininmm number of edges required to produce 
the final parse. 
3 Test Data 
We conducted two experiments. For experimen- 
t1, we parsed a sentence-aligned pair of Span- 
ish and English corpora, each containing 1155 
sentences of Microsoft Excel Help Text. These 
pairs of parsed sentences were divided into dis- 
tinct training and test sets, ninety percent for 
training and ten percent fbr test. The training 
Source Tree Target Tree 
D = volvcr D' = recalculate s,,I,J  
A = Excel E = calcular 
Obj~en A' = Excel I C' = workbook 
B' = values / C = libro 
k , 
B =valores \ae 
F = trabajo 
Excel vuelve a calcular Excel recalculates 
valores en libro de trabajo values iu workbook 
Figure 1: Spanish and English I{egularized 
Parse 2¥ees 
set was used to acquire transfer rules (Meyers 
et al., 1998b) which were then used to translate 
tile sentences in tile test set. This paper focus- 
es on our technique for applying these transfer 
rules in order to translate the test sentences. 
The test and training sets in experiment1 
were rotated, assigning a different tenth of the 
sentences to the test set in each rotation. In this 
ww we tested tile program on the entire corpus. 
Only one test set (one tenth of the corpus) was 
used for tuning the system (luring development. 
~:ansfer rules, 11.09 on average, were acquired 
t'rom each training set and used for translation 
of the corresponding test set. For Experiment 
2, we parsed 2617 pairs of aligned sentences and 
used the same rotation procedure for dividing 
test and training corpora. The Experiment 2 
corpus included the experinlentl corpus. An av- 
erage of 2191 transfer rules were acquired from 
a given set of Experinmnt 2 training sentences. 
Experimentl is orchestrated in a carefld man- 
ner that may not be practical for extremely 
large corpora, and Experiment 2 shows how the 
program performs if we scale up and elilniuate 
some of the fine-tuning. Apart from corpus size, 
there are two main difference between the two 
experiments: (1) the experimentl corpus was 
aligned completely by hand, whereas the Exper- 
iment 2 corpus was aligned automatically using 
the system described ill (Meyers et al., 1998a); 
and (2) the parsers were tuned to the experi- 
mentl sentences, but not the Experiment 2 sen- 
tences (that did not overlap with experinmntl). 
538 
1) A = Excel 
2) B =valores 
C = libro 
v A' = Excel 
B' = values 
.~) 
r C' = workbook 
F = trabajo 
D = volvcr S.IJ.i~ 
4) 
1 E = ealcular 
O b. \]~en l 
2 3 
1)' = recalculate 
1 2 3 
Figure 2: A S('t of %-ansfer Rules 
4 Parses and Transfer Rules 
Figure 1 is a pair of "regularized" parses t br a 
corresi)onding pair of Spanish and Fmglish sen- 
tences fi'om Microsoft Excel hell) text. These 
at'(; F-structure-like dependency analyses of sen- 
tences that represent 1)redicate argument struc- 
ture. This representation serves to neutralize 
some ditfbrences between related sentence tyt)es, 
e.g., the regularized parse of related active and 
t)a,~sive senten(:es are identical, except tbr the 
{i'.ature value pair {Mood, Passive}. Nodes (wfl- 
ues) are labeled with head words and arcs (fea- 
tures) are labeled with gramma~;ical thnetions 
(subject, object), 1)repositions (in) and subor- 
dinate conjunctions (beNre). a. For demonstra- 
tion purposes, the source tree in Figure 1 is the 
input to our translation system and the target 
tree is the outl)ut. 
The t;ransfer rules in Figure 2 can be 
used to convert the intmt; tree into the out- 
1)at tree. These transtbr rules are pairs of 
corresponding rooted substructures, where a 
substructure (Matsumoto et al., 1993) is a 
connected set of arcs and nodes. A rule 
aMorphologieal features and their values (Gram- 
Number: plural) are also represented as ares and nodes. 
consists of o, ither a pair of "open" substructures 
(rule 4) or a pair of "closed" substructures (rules 
1, 2 and 3). Closed substructures consist of s- 
ingle nodes (A,A',B,B',C') or subtrees (the left 
hand side of rule 3). Open substructures con- 
tain one or more open arcs, arcs without heads 
(both sul)structures in rule 4). 
5 Simplified Translation with 
Tree-based Transfer Rules 
The rules in Figure 2 could combine by filling 
in the open arcs in rule 4 with the roots of the 
substructures in rules 1, 2 and 3. The result 
would be a closed edge which maps the left; tree 
in l,'igure, 1 into the right tree. Just as edges of a 
chart parser are based on the context free rules 
used by the chart parser, edges of our trans- 
lation system are, based on these trans~L'r rules. 
Initial edges are identical to transtb, r rules. Oth- 
er edges result from combining one closed edge 
with one open edge. Figure 3 lists the sequence 
of edges which wouhl result from combining the 
initial edges based (m Rules 1-4 to replicate, the 
trees in Figure 1. The translation proceeds by 
incrementally matching the left hand sides of 
Rules 1-4 with the intmt tree (and insuring that 
the tree is completely covered by these rules). 
The right-hand sides of these comt)atil)le rules 
are also (:ombined t;o 1)reduce the translal;iolL 
This is an idealized view of our system in which 
each node in the input tree matches the left;- 
hand side of exactly one transfer rule: there is 
no ambiguity and no combinatorial explosion. 
The reality is that more than one transfer rules 
may be activated tbr each node, as suggested 
in Figure 4. 4 If each of the six nodes of the 
source tree corresponded to five transfer rules, 
there are 56 = 15625 possible combinations of 
rules to consider. To produce tlm output in Fig- 
ure 3, a minimum of seven edges would be re- 
quired: four initial edges derived ti'om the o- 
riginal transfer rules plus three additional edges 
representing the combination of edges (steps 2, 
3 and 4 in Figure 3). The speed of our system is 
measured by the number of actual edges divided 
by this minimuln. 
4The third example listed would actually involve two 
trm~sfer rules, one translating "volver" to "ret)cat" and 
the second translating "calcular" to "calculal;e". 
539 
1) 
2) 
D = volver 
Su~ 
1 E = calcular 
Obj~n 
2 3 
D = volver 
A = Excel E = calcular 
Obj~n 
2 3 
v 
v 
D' = recalculate 
I 2 3 
D' = recalculate 
A' = Excel 2 3 
3) 
D = volver 
A = Excel E = caleular 
B = valores 3 
D' = recalculate 
A' = Excel / 3 
g' = values 
4) 
D = volver 
A = Excel E = calcular 
Ob/~n 
B = vaiores C = libro 
de 
F = trabajo 
v 
D' = recalculate 
A' = Excel \ C' = workbook 
B' = values 
Figure 3: An Idealized Translation Procedure 
6 Best First Translation Procedure 
The following is an outline of our best first 
search procedure for finding a single translation: 
1. For each node N, find TN, the set of com- 
patible transfer rules 
2. Create initial edges for all TN 
3. Repeat until a "finished" edge is tbund or 
an edge limit is reached: 
(a) Find the highest scoring edge E 
(b) If complete, combine E with compati- 
ble incoml)lete edges 
(c) If incomplete, combine E with com- 
patible complete edges 
(d) Incomplete edge + complete edge = 
new edge 
The procedure creates one initial edge 
for each matching transfer rule in the 
database 5 and puts these edges in a 
'~The left-hand side of a matching transfer rule is com- 
patible with a substructure in the input source tree. 
540 
D' = recalculate 
D = velvet 1 2 3 /% 
Sub, i / ~ a !)' = calculate / \ 
/ E = \'4. '+" 
3 again 
D = repeat 
Sabj ~bj 
1 E = calculation 
Figure 4: Multiple \[lYansfer Rules for Each Sub- 
structm:e 
queue prioritized by score. The pro- 
cedure iteratively combines the best 
s(:oring edge with some other comt)al;ilfle 
edge to t)roduce a new edge. and inserts the new 
edge in the queu('.. The score for each new edge 
is a function of the scores of the edges used to 
produce it:. The process contimms m~til either 
an edge limit is reache(l (the system looks like 
it; will take too long to terminate) or a complete 
edge is t)roduced whose left-hand side is the 
input tree: we (:all this edge a "finished edge". 
We use the tbllowing technique for calculating 
the score tbr initial edges. 6 The score tbr each 
initial edge E rooted at N, based on rule/~, is 
calculated as follows: 
1. SCO17.F=I(S) " " F,.c.,~(n) = ~'°.q'~D~(~a ~ ~t N~) 
Where the fl'equency (Freq) of a rule is the 
nmnber of times it matched an exmnple in 
the training corpus, during rule ~cquisition. 
The denominator is the combined fl'equen- 
cies of all rules that match N. 
aThis is somewhat det)cndent on the way these |;rans- 
fer rules are derived. Other systems would t)robably have 
to use some other scoring system. 
Ezperiment 1:1155 sentences 
Norm No Norm 
Total Translations 
Over Edge Limit 
Actual Edges 
Miniature Edges 
Edge Ratio 
Accuracy 
1153 
2 
93,719 
22,125 
3.3 
70.9 
1127 
28 
579,278 
20,125 
1.4.8 
70.9 
Ezpcriment 2:2617 sentences 
Norm No Norm 
Total Translations 
Over Edge Limit 
Actual Edges 
Minimum Edges 
Edge Ratio 
A(:curacy 
2610 
7 
262,172 
48,570 
4.0 
62.6 
2544 
73 
1,398,796 
42,770 
15.5 
61.5 
Figure 5: Result:s 
2, S s ) = s ,o,.(;.l ( S ) - No,.,,,, 
Where the Norm (normalization) t~ctor is 
equal to the highest SCORE1 for any rule 
matching N. 
Since the log.2 of probabilities are necessarily 
negative, this has the effect of setting the E of 
each of the most t)rol)able initial edges to zero. 
The scores tbr non-initial edges are calculated 
by ad(ling u I) the scores of the initial e(tges of 
which they are comt)osed. 7 
Without any normMization (Score(S) = 
SCORE1 (,9)), small trees are favored over large 
trees. This slows down the process of finding the 
final result. The normalization we use insures 
that the most probable set; of transihr rules are 
considered early on. 
7 Results 
Figure 5 gives our results for both experiments 
1 and 2, both with normalization (Norm) and 
without (No Norm). "Total Translations" refer 
to the number of sen|;ences which were translat- 
ed successfully 1)y the system and "Over Edge 
Limit" refers to the numl)er of sentences which 
caused the system to exceed the edge limit, i.e., 
once the system produces over 10,000 edges, 
trm~slation failure is assmned. The system cur- 
7Scoring for special cases is not; included in this paper. 
These cases include rules for conjunctions and rules ibr 
words that do not match any transfer rules in a given 
context (we currently leave the word untranslated.) 
541 
rently will only fail to produce some transla- 
tion for any input if the edge limit is exceed- 
ed. "Actual Edges" reibrs to the total number 
of edges used tbr attempting to translate every 
sentence in the corpus. "Minimum Edges" refer 
to the total minimum number of edges required 
for successful translations. The "Edge Ratio" 
is a ratio between: (1) "Total Edges" less the 
mnnber of edges used in failed translations; and 
(2) The "Minimum Edges". This ratio, in com- 
l)ination with, the number of "Over Edge Limit" 
measures the efficiency of a given system. "Ac- 
curacy" is an assessment of translation quality 
which we will discuss in the next section. 
Normalization caused significant speed-up for 
both experiments. If you compare the total 
number of edges used with and without nor- 
malization, speed-up is a factor of 6.2 for Ex- 
periment I and 5.3 for Experiment 2. If you 
compare actual edge ratios, speed-up is a factor 
of 4:.5 tbr Experiment 1 and 3.9 tbr Experiment 
2. In addition, the number of failed parses went 
down by a fhctor of 10 for both experiments. As 
should be expected, accuracy was virtually the 
same with and without normalization, although 
normalization <lid cause a slight improvemen- 
t. Normalization should produce the essentially 
the same result in less time. 
These results suggest that we can probably 
count on a speed-up of at least 4 and a signif 
icant decline in failed parses by using normM- 
ization. The ditferences in performance on the 
two corpora are most likely due to the degree of 
hand-tuning for Experiment 1. 
7.1 Our Accuracy Measure 
"Accuracy" in Figure 5 is the average of the 
tbllowing score for each translated sentence: 
ITNYu ~ TMSI 
1/2 x (ITNYuI + ITMsl) 
TNZU is the set of words in NYU's translation 
and TMS is the set of words in the original Mi- 
crosoft translation. If TNYU = "A B C D E" 
and TMS = "A B C F", then the intersection 
set "A B C" is length 3 (the numerator) and 
the average length of TNZU and TMS is 4 1/2 
(the denominator). The accuracy score equals 
3 + 4 1/2 = 2/3. This is a Dice coefficient com- 
parison of our translation with the original. It is 
an inexpensive nmthod of measuring the pertbr- 
mance of a new version of our system, hnprove- 
ments in the average accuracy score for our san> 
ple set; of sentences usually reflect an improve- 
ment in overall translation quality. While it is 
significant that the accuracy scores in Figure 5 
did not go down when we normalized the scores, 
the slight improvement in accuracy should not 
be given nmch weight. Our accuracy score is 
flawed in that it cannot account for the follow- 
ing facts: (1) good paraphrases are perfectly ac- 
ceptable; (2) some diflbrences in word selection 
are more significant than others; and (3) errors 
in syntax are not directly accounted tbr. 
NYU's system translates the Spanish sen- 
tence "1. Selection la celda en la que desea 
introducir una rethrencia" as "1. select the cel- 
l that you want to enter a reference in". Mi- 
crosoft translates this sentence as "1. Select the 
cell in which you want; to enter the reference". 
Our system gives NYU's translation an accu- 
racy score of .75 due to the degree of overlap 
with Microsoft's translation. A truman reviewer 
wouhl probably rate NYU's translation as com- 
pletely acceptable. In contrast, NYU's system 
produced the following unacceptable translation 
which also received a score of .75: the Spanish 
sentence "Elija la funcidn que desea pegar en la 
f6rmula en el cuadro de di~logo Asistente para 
flmciones" is translated as " "Choose the flmc- 
tion that wants to paste Function Wizard in the 
formula in the dialog box", in contr,~st with Mi- 
crosoft's translation "Choose the flmction you 
want to paste into the tbrmula fl'om the Func- 
tion Wizard dialog box". In fact, some good 
translations will get worse scores than some 
bad ones, e.g., an acceptable one word trans- 
lation can even get a score of 0, e.g.,"SUPR" 
was translated as "DEL" by Microsoft and as 
"Delete" by NYU. Nevertheless, by averaging 
this accuracy score over many examples, it has 
proved a valuable measure for comparing differ- 
ent versions of a particular system: better sys- 
tems get better results. Similarly, after tweak- 
ing the system, a better translation of a partic- 
ular sentence will usually yield a better score. 
8 Future Work 
Fnture work should address two limitations of 
our current system: (1) Bad parses yield bad 
transihr rules; and (2) sparse data limits the size 
of our transfer rule database and our options for 
542 
applying transfer rules selectively. To nttack the 
"bad parse" problem, we are eonsideriug using 
our MT system with less-detailed parsers, since 
these parsers typically produce less error-prone 
output. We will have to conduct exl)erimcnts 
to determine the minimum level of detM1 that 
is needed, a 
Previous to the work reported in this paper, 
we ran our MT system on bilinguM corpora in 
which the sentences were Migned manuMly. The 
cost of manuM aligmnent limited the size of the 
corpora we could use. A lot of our recent MT 
research has bo.en tbcused on solving this sparse 
data prol)lem through our develoi)ment of a sen- 
tence alignment progrmn (Meyers et al., 1998a). 
We now have 300,000 automaticMly aligned sen- 
tences in the Microsoft help text domain tbr fu- 
ture experiineni;s. In addition to provi(ting us 
with many more transfer rules, this shouhl Mlow 
us to colh'.ct transfer rule co-occurrence infor- 
mation which we c~m then use to apply tr;mstbr 
rules more effectively, perhaps improving trans- 
b~tion quality. In a preliminary experime, nt a- 
hmg these lines using the Experiment 1. tort)us, 
co-occurrence information had no noticeable ef 
feet. However, we are hot)eflfl that flltm'e ex- 
t)eriments with 300,000 Migned sentences (300 
tinies as nnlch data) will 1)e more successful. 

SOne could set u 1) a contimmm from detailed parser- 
s like Proteus down to shallow verb-group/noun-grouI) 
recognizers, with the Penn treetmnk based parsers ly- 
ing somewhere in the middle. As one travels down t, he 
eonLinlmIn t;o t;he lower detail parsers, tim error rate nat- 
urally decreases. 

References 

Robert J. Bobrow. 1990. S1;~Ltistical agenda 
parsing. In I)ARPA Speech and Lang'uagc 
Workshop, pages 222-224. 

Peter Brown~ Stephen A. Delb~ t)ietra, Vin- 
cent J. Della Pietra, and Robert L. Mer- 
cer. 1993. The Mathematics of Statistical 
M~zchine 'h'anslation: 1)arametcr Estimation. 
Computational Lin.quistics, 19:263-312. 

Sh;~ron A. Caraballo and Eugene Chm'niak. 
1997. New figures of merit tbr best-tirst prot)- 
M)ilistie chart parsing. Computational Lin- 
guistics, 24:275-298. 

Eugene Ctmrniak, Sharon Goldwater, and M~rk 
Johnson. 1998. Edge-Based Best-First Chart 
Parsing. In Proceedings of the Sixth Annual 
Workshop for Very Lawc Corpora, Montreal. 

Mahesh V. Chitrao and RMph Gris}unan. 1990. 
St;~tisti('al pnrsing of messages. In \])AIIPA 
Speech and La'n,g'uagc Workshop, pages 263 
266. 

Annn Sggvall ltein. 1996. Pretbrence Mecha- 
nisms of the Multra Machine %'ansb~tion Sys- 
tem. In Barbara H. Partee and Petr Sgall, 
editors, Discourse and Meaning: Papers in 
11onor of Eva 11aji~ovd. John Benja.mins Pub- 
lishing Company, Amsterdam. 

Y. Matsumoto, H. Ishimoto, T. Utsuro, and 
M. Nagao. 1993. Structural Matching of 
Parallel Texts. In 31st Annual Meeting of 
the Association for Computational Linguis- 
tics: "Proceedings of the Uo~@rencc". 

Adam Meyers, Roman Ymlgm'ber, a.nd Ralph 
Grishman. 1996. Alignment of Shared 
Forests fi)r BilinguM Corpora. In Proceedings 
of Coliw.I 1996: The 16th International Con- 
fercncc on Computational Linguistics, l)ages 
460 465. 

Adam Meyers, Miehiko Kosak~, and Ralph Gr- 
ishman. 1998m A Multilingual Procedure 
for Dict;ionary-B;~sed Sentence Aligmnent. In 
Proceedings of AMTA '98: Machine Transla- 
tion and th, c ht:fo'rmation Soup, t)~ges 187. 
198. 

Adam Meyers, R,om~m Ym~g~rber, Ralph Gr- 
ishmml, Cntherine Macleod, mM Antonio 
Moreno-S~mdow~l. 1998|). l)eriving ~l~:a.ns- 
fin: Rules from Domimmce-Preserving Align- 
ments. In I)'rocccdim.ls o.f Coling-A CL98: Th.c 
171h International Conference on Computa- 
tional Ling,uistics and the 36th, Meeting of the 
Association for Computational Linguistics. 

Sergei Nirenlmrg mM Robert E. l~:ederking. 
1995. The Pangloss Mark III Machine 'l?nms- 
lt~tion System: Multi-Engine System Archi- 
tecture. Te(:hnical report, NMSU Oil,L, USC 
ISI, ;rod CMU CMT. 

Andrew Way, Ian Crookston, and Jane Shell;on. 
1997. A Typology of ~IYanslation Prol)lems 
for Eurotra Translation Machines. Machine 
\[l}'anslation, 12:323 374. 
