Two Types ot' Adaptive MT Environments 
Sergei N1RI,;NI{IJR(~ R()l)crl FI{EI)I~;I{KIN(; 
Center for Machine Translation Center Im" Machine Translation 
Carnegie Mellon University Carnegie Mellon University 
Pittsburgh, PA, USA I:qttsburgh, PA, USA 
l)avid FAI{WI,',I,I, 
Computii~g Research Imb 
New mexico State University 
I.as Cruces, NM, liSA 
Yorick WII,KS 
l)cpartmcnt of (;tlmputcr Science 
University o1' Shcftiehl 
tJ I( 
ABSTI~ACT 
A nurnher of propos'ds have come up hi re- 
ten! years for hybrhlizitlhm of MT. Currcnl MT 
inojects - both "pure" and hyhrhl, both prc- 
do,ninantly leclul,.llogy-orienlcd and scienlilic 
(including those currently funded hy NSF) ;ire 
single-engine projecls, CalJahle of one parlic- 
ular type of source text analysis, one partic- 
ular rncthod of lh/ding target lan~;lia~,e corre.- 
spondences for source language elcmenls and 
one prescribed metho(I cd gcncralhlg tile Iitr- 
gel language text. While such pr(~jccls can be 
quite useful, we believe thnt il is lime tc~ make 
tile next step in lhe desiy, n cd machine Ii.{ins- 
httion systems and to move lOw;id i~dii\])live, 
mulliplc-crlgine syslems. We describe Ihc ar- 
chitecture of an adaptive multi-engine MT sys- 
tem which uses each of the engines under tile 
circumstances which are most favorable for ;Is 
success. 
I. Multi-Engine MT Architech.'e 
P, eccn! years have wilnessed a shifl in tile hahmcc uf sci- 
entific and technological efl'orls in the nrea {~t" nmchinc 
translation. All tile latest mcthc~doh+gic;tl m/vcllics in this 
field ;u'c essentially technology-c.,ricnted and do not aim 
ill advltnch\]g our knowledge itl'Joul either basic tnecha- 
IliSlllS of text coralschess;on and production Of COIll\])tllel + 
models shnHhlling stich i11e.challiSlllS. 
Tile lwo lllOSl recently popular techncd o~,{ical parndit, ms 
ill machine translation--- e×ample-I'~ased Iranslalion 
(EBMT) and stalisliCSdlased transhlthm (SIIM'f) ---re- 
quire linguistic knowledge only :is an aflerlhollghl. While 
the represenlatives of the above paradigms are still al lhc 
stage, of e.ilher building toy systems (e.g., Furuse and litht, 
1992; McLean, 1992,Jones, 1992, Maruyama and Wltlan- 
,aim, 1992) or struggling with tile natural constraints olal> 
proaches that eschew the Sttldy ol" langual.,e ;is such (e.g., 
Brown et at., 1990), .it number of llropi`'sals have come up 
lor some hybridization oF M'I: \[n some such .aplnO',tches, 
Corl)llS analysis is tised ('Ill" ltlllhl\]:{ analysis ;lid Ii.{uIsfcr 
grammars (e.t;., Su and ('hang, 1992). Ill olhcrs, a stan- 
dlu'd tr:msfcx-I'~L~ed aPl/rtmch (TBMT) is followed usiny, 
hadilh/nal analysis and generalhm technhlueS bul havin!,, 
a IranslEr component Imscd on aligned I,ilingual corp(ira 
((lrishnmn and Kosnka, I")92). Slillotherx, use slalisli- 
cal illforlll;llh`'ll ;IS Ihc source of preference axsil~lllllC.Iil 
durin!; lexl dismnbig, ualion (e.g., tile OUlIHle. presenlcd ht 
l.chnmnn and ()ll, 1992). Slalislical modelinp can be 
used al some sb:l,ges of a knowled~;ed}ased MT (KBMT) 
system (see, e.g., t lehnreich, 1'-)94). 
Current MT projt!cts - Iloth "pure" lind hybrid, 
bt~lh Fu+cdmnhmntly techncdtB, y-c~rienled ill/d scientific arc 
shl!,Jc-cn~,,inc projccls, Calmblc of one particular lype of 
SOUl'CC lest analysis, one particular Inelhod of finding lar- 
gel Iiln~uage cc, rresl',Cmdenccs for source hmgua,ge ele- 
meuts and one prescribed nlethod of generating Ihc large\[ 
hmguaI.,e lest. While such projects can be quite useful, 
we t:,elieve Ihal il is Ihne to make Ihe next step in the de- 
SigIl of llliIC\]l\]lll2 ll'\[lllxlilliOll systems iltld l(\] illOVe loward 
adIq`'live, mulliple-enghm syslems. 
Praclical MT syslems are lypically developed lor a par- 
licular Icxl type (e4,., wc;llhcr rcl)t)rls , linancial news ar- 
ticles, scientific nh, stracls) anti for a parlicular end use -- 
e.l'., assinlilalion or dissemhlItficm o1 \]llfOl+lllillh)n. Sl/0- 
cial cases, such as lranslalhlg till tlpdaled version of a pre- 
viously llilllS\[\[llt.:(I \[cx\[, ilt)Olll/(\] ill Ihc rcid-worl,,I \])faClilTc.. 
Gains in t:,Ull'~Ut quality and eflicicncy can I;e expected if; 
IllllthilIc hanslalhm ellvhonlllelll Call he made. I\[:, iRlil\[)l It) 
a task prolilc. Thus, \[t'u cxmnple, fc, r lransl;lling nbslrltclS 
of scientific arlicles ill order to sclccl jusl lhose Ihal are 
of particular inlercs t to a customer, it statistics-based ap- 
prc, ach might he mosl apprc, priate. Extlmple-based trans- 
htlion seems It, be most l/ronlisinE lor It+ansi;thiS new 
versions of previously translated (hlcIIillelllS. This cor- 
respondence hctwecn lechnique, input texl type and end 
use (or Olllptlt lext lype) provides ItlHher nlotivatioll for + 
moving lowat(I adaptive, nnnllilfle-engine systems. 
Wc perceive lwo alqu'oaches to adaplivily in MT. P, olh 
llreSupl~OSe an MT environment ill which a nunlher tff 
MT engines are present -- for irlstlmcc, one (t)r morel) 
each of KBMT, SI;MT, F.BMT and TBMT cn~ines can 
125 
he used. hi one of tile approaches all availahle engines 
are "unleashed" on an input text and file liJml otntpul is 
assembled from the best rex! segmeras, i,'respective of 
which engine prodtmcd them. We. call this approach 
lhe Best Otttput Segment (B()S) approach, hi another 
approach a heuristic "dispatcher" decides which of the 
available engines holds tile highest p,-omise for it given 
input text and then assigns the job to that engine. This is 
tile DiSlmtcher-Based (DB) approach. The B()S approach 
involves more processing bill ~lllows an ,.7 posreriori se- 
lection of the best results. Tile DB approach saves cycles 
but relies on heuristic ,,., priori selection of the best oulput. 
hi this latter case, lhe quality of the dispalchermodule is 
crucial, but additionally, the DP, approach expecls each c,f 
the component engines to be of rather Irish quality, since 
they would not (as is the case in lhe BOS apF, rOach) he 
"bailed onl" hy olher engines in case of failure. 
hi what lk-dlows we hrielly describe our lirst experiment 
with tile B()S approach and discuss tile requirements for 
tile DB approach. 
2. The Best Output Segment Apprtmch to 
Adaptlvity 
()ur B()S Itpproach experiment was cmried out lor a Span- 
ish - English Iranslation sel-u F, in the fran/ework of Ihe 
Pangloss MT project (F'angloss, 1994)and used Ihrce. MT 
engines -- KBMT, EBMT, and TBMT. 
Tile KBMT engine we used was tile. mainline engine 
of tile Pangh./ss system, a lradilional KBMT environment 
described in some detail in (Pangloss, 1994). It was im- 
portant for tile BOS experiment that this engine genernlcd 
an internal quality rating for each OUtlmt segment it pro- 
duced. 
The tmsic idea of EBMT ix simple (cf. Nagao, 19g4): 
an input passage S is comlmred with the sourcc-I:mguage 
"side" of a bilingual lexl archive, where lexl passages are 
slored witll their Iransllitions ill|() It flu'gel langtmge (or a 
set of such). TIle "closesl" matcll, passage S' is selecled 
and lhe lranslation of lhis clos'ast malch, the passage 7" is 
accepled as tile IranslaliOn of S. Our EBMT engine iiscd 
a 100MB bilingual Spmfish - English archive of UN cdti- 
cial documents, hi preparation fc, r processir~g, die archive 
was idigned at tile sentence level. Tile lnalching of input 
passages with the Spanish side of the archive was allowed 
to be inexact. Penalties were assessed for omitted and ex- 
Ira words, word occurrences ill different rnorphological 
forms and differences in word order. The English siring 
lranslaling lhe best Spanish archive candidale wlts then 
lound in tile English sentence aligned with tile Spanish 
sentence in which lhe hest match candidme appea,',ad. A 
.Spanish - English MRD win; used in determining Iransht- 
lions of individual words inside the candidale segmenis. 
A special ronlinc then calculated lhe expected quality o1 
the resulting Iranslatiorl, which helped .;it lhe restlll inle- 
gralion stage of nnllti-engine MT syslenl operaliOn. ()ur 
EBMT approach ix described in Nirenln.~g el al., 1993 
and Nirenburg ,at al., s.bmiHed). 
Our transfer systenl was very simple. It was Ilascd on 
direct Icxical substitution fo English words and phrases 
for Spanish words and phrase, forlilied wilh n/orpholog- 
ical analysis at/d synlhesis modifies. Tile process relied 
on ii tltllllber of dalabases - it Slmnish - English MRD, 
lhe lexicons used by ll/e KBMT engine, a large sel of 
user-generated bilingual glossaries as well as a gltzetleer 
and It list of proper and orglll/izaliOll names. The user- 
gerler.:llcd glossaries for our experiment corllained aboul 
174,000 entries. Glossary enlires conlained variahles to 
allow feature matching and indices U.i link the parts of 
phrasal elllries Ilia\[ translated Olle anolller. Fof illS\[ante, 
lhe following glossary enlry 
absolver<l> ~t <dop:2> de 
--po::s:2> promesa 
:=> 
release<l> <dop:2> from 
<po:;s : 2- prolfli~31 ~ 
can help to generate such English sentences as 
I r~9\].ease yo~l froln your promise; 
He released me from my promise; 
You will be releasing her from 
her promisa ; 
i~ t c . 
In tile rule above dop stands for"dhecl object pronoun" 
and poss for "possessive." "l, fl-,les of feIflure correspon- 
dences were prepared to make the translation possible. 
Note that in many CltSCS Spanish features and English 
fealurcs were quile differenl (rlot;ih, ly, for verbs). The 
rltn,nbers in "mgtthlr brackets are indices which show the 
mo,pht+logical synthesizer which word Io put in a par- 
ticular form at generation time. In lifts expe,iment we 
used variables for the lollowing word classes: proper 
IlllllICS, such as imlividual, ct'mlplllly and ill:tee Illlllies; 
litltllllcrs itl/d the vltriotls classes of prollotlns -- persorml, 
possessive, rellcxive, direcl ohjccl, indirecl olljecl and 
possessive al~st)hile. 
2.1. Combining Results 
The crux of tile B()S melhod is combining, results from 
indivhlual engines. A clmrl data strllCltlrc wil~ Itge(\] tO 
COlllbinc resells l\]t)lii Ihc individual engines. Bclbre the 
lranshffion process, the edges of lhe chart were made to 
correspond to indivkhml words in the input. New edges 
are added It) the ch:lrl tilrougtl tile operation of the throe 
MT erlgines labeled wilh tim Iransration of a scgmenI of 
tile inpul siring and indexed hy this segment's heginning 
and end positions. The KBM'I\[ and EBMT engines also 
carried a quality score for each ioulpul elemenl. 
After all lima engines finished their work il is lleC(b;- 
126 
sary lo lind the sequence of transhltion candidates vehich 
~0 cover the input string as densely as possible (so Ihat 
there is a Iraiislalion for ak I\]laI/y source lexl elements as 
possil',le); b) use the "hesl" of lhe available canditlales. 
q~'~ lind the best candidates three heuristics were used 
a) intern'd quatily ratings produced by the KP, MT and 
EBMT engines; h) stalic relative qu;dity assessmcnl of 
the protitular engines wc used and c) the length of lhe 
translation segment (the longer, Ihe hetter). Enhancing 
lhe quality of lhese hemistics antl generally tinding more 
Sol)hislicated ways of combining timlings of individual 
engines is the most important direction of improvenlenl 
of ollr BOS system. 
The chart walk algorithm l)roducing the final result of 
lhe B()S system used lhe above heuristics. The algorilhm 
uses dynamic programming to lind the Ol~linmI cover (a 
cover with the best cnmtllative score), aSS\]lining correct 
component qualily scores. 11 ix dcscrihed in some detail 
and illustralcd in Nirenhurg and l:rcdcrking, 1994 and 
Frederking an(I Nirenburg, sttl,tfftted. 
3. The l)isl)atcher-Based AI)l)roach to 
Adaptivity 
In this apfuoach, ,'t dispalcher nlodule ix used to break up 
the input text into segments and assign each segmcnl Io 
one or another o1' tile avaihtble MT engines. Among Ihc 
possible diagnoslics l~'~r the dispatcher are: 
* q~ype of translation -- whether the rcstilt of lransht- 
lion is intended for disscmin:ttion or for assimihttitm; 
whether a complete lranslaticm is nccdetl or an ab- 
st,act or even a simple categorizalitm of a Iext (e.g., 
as a text Ihat is iml~ort;mt CllOIIgh Io be Iranshdcd in 
its entirety). 
. Availability of parallel lext iu a parlicuhu domain 
'm(I on a Imrticular topic. This ix lhe crucial cnal'fling 
condition 15r EBMT and SP, MT. 
t, Amoltnl of ambiguity in Ihc source passage, hoth 
in tile source language itself and vis-a-vis a I;irget 
language. The smaller the tlegrce of anlhilmily, Ihe 
more attractive the KBMT approach. 
.t. Size and quality of available KBMT resolnces (on- 
tology, lexicons, etc.). 
The work on the dislmtcher, thus, includes a) evalual- 
ing tile translation contcxL with rcspccl to tile fore" crilc- 
ria above and 1"0 pulling Iogelher a decision mechanism 
which will establish the relative ,:q:,propriatcncss of each 
of tile available engines lbr treating all input passage in a 
given context. All additionaI important parameter in Ihe 
operation of tile dispatcher is determirung the most ap- 
prol)riate size of input passage to be dispatched It) an MT 
engine. Since tin entire input text c~.tn t'~C processed hy a 
combination of MT engines, it is necessary to maximize 
tilt: cxpcclcd quality of Otllllllt OVCf ;I vark.'Ay of possil'~le 
ways of"chunking" tile input Icxl for processing. This has 
some similarity with the chart walk in file B()S alqm)ach. 
The disimtchcr will unsc an additional set of diagnostics 
dctcrlllillcd by file slntlClllre of Ihe spccitic MT engine. 
Tim dcvelt)l'llllel/I ()\[lhe:.;c dispatcher heuristics - ill (,lhcr 
words, how the dispalchcr is to be h-ained (see below) .... 
is a key l)Oinl of tile \[l\[of~ose(I research. A prelinunary 
analysis elthese spccilic tliagnoslic heuristics, orderc(I by 
Ihc parlicuhu" cngil\]c, follows. 
An additional tliagnoslic heuristic lot SBMT inspects 
Ihc frcqucrlcy olc, ccurfencc of each iil(livi,.Itt;ll input slrillg 
ileal\] in the corl)tts. The greater the frequency of the items 
c()ntaincd ill the lcxt, tile glCaler the likelihood lhal tile 
SRMT engine will produce \[,,ood tlualily OUtlml. 
The ahovc heuristic will also serve tile EI',MT engine. 
A heuristic uscfut spccilically for EBMT is the ~llllOtll\]t of 
overlap (if ;ill ill\]Hit IeXt with a (lOclll/\]et\]l ahcady in llle 
source lan,t,,lmgc si(Ic of the bilingual archive. 
The diagnostics lot tile "\['BMT and KBMT al)inoaches 
moslIy check Ihc coverages of approprialc slalic knowl- 
et\[~c SOUlCCS - - ~fltllllll;.trs ,:llltl lexicons. 
Tim diagnostics proposed above vary in cosl, both in 
tt.'l'lllg c,f developing the procedures and in tel'ill,'-; Of Ihcir 
colnplll;ltional conlplcxity. Rehllively inexpensive are 
tliat,xlt)stics h;Isctl oil recognizing il\]dividual terms or pal- 
toms in the inptlt (e.g., chccki,'lg tile availahilily of ilcms 
ill a lexicon t\]r it corplls, chcckhlg tile lenglh of segtllCllt:.,, 
checking for local sct\]llellcing p'alterns of forms). Soil|e- 
what Illore cxpcnsivcarc diagnostics based on a.',:signnlenl 
of catc,t,,orics to forlns. \[1 is screndipilous, howevcq thai 
tile more cosily tli'lgnosfics are generally related to mi- 
lial stages of pnocenxin t, nccessgry ill tilt)st cnghles. This 
opc'.ns a pt~lcnlial I(u inlerleaving Ihe processing by indi- 
vidual engines with lhc operation of tile disp;llchcr. 
4. Fulure Work 
"I'he questions of how Io optimize the colnlfinalion el'evi- 
dence in Ihe P,( )S al~pro;lch and how 1o trail1 Ihe, disp:tlcher 
ill the DB apl:uoach ale very close 1o ~\[l key (llleSlion in 
mc~(lern MT: how an M'F system is to be evahl\[llc(I (even :ix 
a small-scale proof of concept). We. plan an experh/lental 
smtly to ilnprove, the procedure for the combinalion of ev- 
idence from the individual engines in tile B()S approach, 
which will include a cOIlll)llriSOll of the rcstllts of our sys- 
tem with htlm;tn j\]IdgmelltS and sill',sequent inodilication 
of the sy.,;Icnl ba.'-;ctl on this feedback. Wc also intend to cx- 
pcrilncnt with ;I training schcdtllC h,y which the disl!atchcf 
COllld be trained over 1ox{ samples, hy Irying potentially 
r;tlldOll/ltg.'.;iglIlllCnts of texl parts to II\]OdtlICS \[|l\]l.I then see- 
ing which assignment regimes produce the host results. A 
v:uiant on this would I',c hul\]lttn text "tttgt\[if~g" by intu- 
itions about tile texl type (where the human lagged it hy 
127 
tile module type that he considered would be needed; this 
would be essentially a difficulty rating the text a priori). 
and again a,;sessing this against system results. As the 
size of such an experiment carl be quite signilicant, we 
envisage the use of some form of qtmsi-automalie quality 
scoring for MT of the sort proposed recently by Henry 
Thompson and his colleagues (e.g., Brew and Thompson, 
1994). 
slalislics-orienled machine translation. Prbceedings of 
TM1-92. Montreal. 249-62. 
Wilks, Y., Fass, D., Gt,o, C-M., McDonald, J., Plate, T., & 
Slalor, B. 1990. Providing Machine Tractable l)ietionary 
Tools. Machine 7)'anslalion, 5:2, 99-15 I. 
References 
Ben Ari, D. M. Rimes and D. Berry. 1988. Tr:mshttional 
Ambiguity Rephrased. Proceedings of the Second Inter- 
national Conference on Theoretical mid Inethotlological 
Issues in Machine T,'anslalion. Pittsburgh, June 1988. 
Brew, C. and 11. Thompson. 1994. Automatic Evalua- 
tion of Computer Generated Text: A Progress Reporl on 
the TextEval Project. F',oeeedings of HLrqX)4, Princeton, 
Mm-ch. 
Brown, P., Cocke, J., Della Pietra, S., Della Pietra, V., 
Jelinek, K, Mercer, R.L. mid Roossin P.S. 1990. A sta- 
tistical approach to language translation, Comlmtatiomd 
Linguistics. vol 16, 79-85. 
Fnrnse, O and H. ikla. 1992. An Example-Based Method 
for Transfer-Driven Machine Translation. Proceedings of 
TMI-92. Montreal. 139-50. 
Grishman, R. and M. Kosaka. 1992. Comhining ralio- 
nalist and empiricist approaches Io mqchine translalion. 
Proceedings ofTMI-92. Montre,'tl. 263-74 
lsabelle, P. anti L. Bourbeau. 1985. TAUM-AVIATI()N: 
Its Technical Features and Some Experimental Results. 
Computational Linguistics, 11 : 1 g-27. 
Jones, D. 1992. Non-hybrkl example-based machine 
translation architectures. Proceedings of TMI-92. Mon- 
Ireal. 163-71. 
Lehmann, H. and N. ()It. 1992. Translation relations and 
the combination of analytical and statistical unetho,,Is in 
Itlachhle trarlslation. Proceedings of TMI-92. Monlrcal. 
237-48. 
Maruymna, H. and H. Waianabe. 1992. Tree cover search 
algorithm for example-based translation. Proceedings of 
TMI-92. Monlreal. 173-84. 
McLemL I. 1992. Example-based machine translation 
using connectionisl matching. Proceedings of TMI-92. 
Montreal. 35-43. 
Nirenburg, S., J. Carbonell, M. Tomita and K. Goodman. 
1992. M~lchine Translation: A Knowledge-Based Ap- 
proach. San Mateo, CA: Morgan Kaufmann. 
Su, K-Y. and J-S. Chang. 1992. Why corpus-based 
128 
