Learning to Select a Good Translation 
Dan Tidhar and Uwe Kiissner 
Technische UniversitAt 13erlin 
Fachbereich Informatik 
Franklinstr. 28/29 
D-10587 Berlin 
Germany 
{ukl dan}~cs, tu-berlin, de 
Abstract 
Within tile machine translation system Verb- 
mobil, translation is 1)ertbrmed simultaneously 
1)y four indel)endent translation lnodules. The 
\['our competing l;ranslatiol~s are coati)|ned 1)y a 
se,\[e('tion module so as to forln a single opti- 
mal outlmt for each intmt utterance. The se- 
lection module relies on confidence values that 
are delivered together with each of the alter- 
native translations. Sin(:e the (:onfidence val- 
ues are computed t)y four independent mod- 
ules that are flmdanlentally difl'erent from (me 
another, they are not dire(:tly (:oml)arat)le and 
ne, ed to l)e rescaled in order to gain (:onq)arative 
signiticance. In this pat)er we describe a ma- 
chine lecturing method tailored to overcome this 
difficulty l)y using offline hmnan thedback to 
determine an at)prol)riate confidence res(:aling 
scheme. Additionally, we des(:rit)e some other 
sour(:es of information that are used tbr select- 
ing 1)el;ween the comt)eting translations, and de- 
scribe the way in which the seh',ction t)rocess 
relates to quality of service specifi('ations. 
1 Introduction 
Verbmobil (Wahlster, 2000) is a speech to 
speech machine translation system, aimed at 
handling a wide range of spontaneous spee('h 
phenomena within the restricted domain of 
travel t)lanning and at)pointment sche(hfling 
dialogues. For the  1)airs English- 
German an(1 Gerinan-English, tbur different 
translation methods are applied in parallel, thus 
increasing the system's robustness and versa- 
tility. Since, exa(:tly one translation should t)(; 
t)roduce(1 for each int)ut utterance, a selection 
1)rocedure is necessary. In order to benelit more 
from this diversity, the alternative translations 
are furthermore, combined within the t)omld- 
aries of single utterances, so as to form \]low corn- 
pound translations. Combining translations 
t'ronl different sources within a multi-thread MT 
system has already proved beneficial in the past 
(Frederking and Nire, nlmrg, 1994). Our i)resent 
work diflhrs fl:om the work reported in there ill 
several ways (at)art from the trivial fact that 
we use 'four heads' rather than three). Firstly, 
we attempt to investigate a systematic solution 
to |;11(; incolnparability of the various confidence 
values. Secondly, as we deal with speech to 
speech rather than text to text translation, dit'- 
ti;renl; segmentations for each given input string 
are allowed, lnaking the segment combination 
process signitic~mtly ulore COml)licated. 
1.1 Incomparability 
Eat;h translation lnodule calculates a confidence 
value tbr each translation that it; produces. 
However, since the w~rious translation methods 
are flmdalnentally difl'erent from one another, 
the resulting contidenee values cannot t)e di- 
re(:tly (:omt)ared across modules. Whereas we 
do assmne a gener~fl (:orrest)ondence between 
confidence wflues and translation quality within 
each one of the mo(hfles, there is no guaranty 
whatsoever that a high value delivered t)y a cer- 
tain module wouM indeed signify a l)etter trans- 
lation when comt)ared with another value, even 
a much lower one, which was delivered l)y an- 
other module. An additional step needs to 1)e 
taken in order to make the confidence wflues 
comparable with one another. 
1.2 Working Hypotheses 
It should be noted that one of our working 
hypotheses, namely, that coniidence values do 
generally reflect translation quality, also (:om- 
t)ensates to ~t certain extent tbr tile lack of a 
wide range theory of translation, according to 
which translations of difli;rent sorts could be 
mlanimously ewduated. The task of evaluating 
843 
translation quality is non-trivial also for human 
annotators, since the applicable criteria are di- 
verse, and at the absence of a comprehensive 
translation theory, very often lead to contra- 
dicting conclusions. This difficulty is partially 
dealt with in section 4.1 below, but tbr practical 
reasons we tend to accept the need to rely on 
human judgment, partially theory assisted and 
partially intuitive, as inevitable. Another pre- 
supposition that underlies the current work is 
that the desirable rescaling Call be well approx- 
imated by means of linear polynomials. This 
assumption allows us to remain within the rel- 
atively friendly realm of linear equations (al- 
beit inconsistent), and reflects two basic guid- 
ing principles: firstly, that tile rescaling is mo- 
tivated by pragnlatical needs, rather than by 
descriptive aspirations, and secondly, that it 
should not contradict the presupposed correla- 
tion between confidence and quality within each 
module, which implies that the rescaling func- 
tions should be monotonous. 
2 The Various Translation Paths 
The Vcrbmobil system includes four indepen- 
dent translations paths that operate ill paral- 
lel. The input shared by all paths consists 
of sequences of annotated Word Itypothcscs 
Graphs (WHG), produced by the speech rec- 
ognizer. Each translation inodule chooses in- 
dependently a path through the WItG, and a 
possible segmentation according to its gram- 
mar and to the prosody information (Buckow 
et al., 1998). This implies that even though 
all translation modules share the same input 
data structure, both the chosen input string and 
its chosen segmentation may well differ across 
modules. This section provides tile reader with 
very brief descriptions of the different trans- 
lation subsystems, along with their respective 
confidence value calculation methods. 
• The ali subsystem implenlents an exam- 
pie based translation approach. Confi- 
dence values are calculated according to the 
matching-level of the input string with its 
counterparts in the database. 
• The stattrans (Ochet al., 1999) sub- 
system is a statistical translation system. 
Confidence values are calculated according 
to a statistical  model of the target 
, in conjunction with a statistical 
translation model. 
• The syndialog (Kippct al., 1999) subsys- 
tem is a dialogue act based translation sys- 
tem. Here the translation invariant con- 
sists of a recognized dialogue act, together 
with its extracted propositional content. 
Tile confidence value reflects the probabil- 
ity that tile dialogue act was recognized 
correctly, together with the extent to which 
the propositional content was successflflly 
extracted. 
• The deep translation path in itself con- 
sists of nmltiple pipelined nlodules: lin- 
guistic analysis, senlantic construction, di- 
alogue and discourse semantics, and trans- 
fer (Emele and Dorna, 1996) and gener- 
ation (Kilger and Finklcr, 1.995) compo- 
nents. The transfer module receives dis- 
anlbiguation information from the conte.xt 
(Koch et al., 2000) and dialogue modules. 
The linguistic analysis part consists of sev- 
eral parsers which, in turn, also operate ill 
parallel (Ruland et al., 1998). They include 
all HPSG parser, a Clmnk Parser and a 
statistical parser, all producing data struc- 
tm:es of tile same kind, namely, the Verb- 
mobil Inter:face Terms (VITa) (Schiehlen ct 
al., 2000). Thus, within the deep process- 
ing path, a selection problem arises, simi- 
lar to the larger scale problem of selecting 
the best translation. This internal selec- 
tion process within the deep path is based 
on a probabilistic VIT model. Confidence 
values within the deep path are computed 
according to the amount of coverage of the 
input string by the selected parse, and are 
subject to modifications as a byproduct of 
combining and repairing rules that oper- 
ate within the semantics mechanism. An- 
other source of intbrmation which is used 
for calculating the 'deetf confidence val- 
ues is the generation module, which es- 
timates the percentage of each transfered 
VIT which can be successfiflly realized ill 
the target . 
Although all confidence values are finally scaled 
to the interval \[0, 100\] by their respective gen- 
crating modules, there seems to be hardly any 
reason to believe that such fimdamentally dif- 
844 
ferent calculation methods would yield magni- 
tudes that aye directly comi)aral)le with one an- 
other. As expected, our experience has shown 
ttsat when confidence values are taken as such, 
without any further modification, their compar- 
at|w; significance is indeed very linfited. 
3 The Selection Procedure 
In order to improve their coml)arative signifi- 
cance, the delivered confidence wflues c(s), tbr 
each given segment s, arc rescaled by linear 
flmctions of the tbrm: 
a-c(,s) + b . (1) 
Note that each inI)ut utterance is decomt)osed 
is|to several segments indet)endently , and hence 
t)otentially differently, by each of tim translation 
I)aths. The different segments arc then cora- 
l)|ned to tbrm a data structure which, by anal- 
ogy to Word Itypotheses Graph, (:ass be called 
~l~'av, slation Alternatives Graph, (TAG). The size 
of tiffs graph is bound t)y 4 '~, whi(:h is reached if 
all translation paths hat)pen to (:hoose an idea> 
tical partition into exactly n segments. The 
following vectorial not~tion was adot)ted in or- 
deY to simpli(y the simultaneous reference to all 
tYanslation t)aths. The linear coefficients are 
represented by the following tbur-disnensional 
vectors: 
~, ~ O"syndial°g i; = l)'s'Y'ndial°9 
(1,start,tans ~)~tattra'n.s 
adccp bdeep (2) 
Single vector comt)onents cms then be referred 
to by sinq)le t)Yojections, if we ret)Yenent the dif 
ferent translatiols paths as orthogonal refit vec- 
tors, so that .~ denotes the vector torrent)ending 
to the module by which s had been generated. 
The normalized confidence in then represented 
by: 
(a. + (3) 
In order to express the desirable fav<)ring of 
translations with higher input string coverage, 
the COml)ared magnitudes are actually the 
(rescaled) confidence wflues integrated with 
respect to the time axis, rather than the 
(rencaled) confides, co values as n.ch. Le|; I1 11 
be tim \]ength of a segment .s of the input 
stream, in milliseconds. Let SEQ be the set of 
all possible segment sequences wil;lfin the TAG, 
and Seq E SEQ any particular sequence. 
We define tile normalized confidence of 
Seq as tbllows: 
s~Seq 
Tlfis induces the following order relation: 
Based on this relation, we define the set B of 
best sequences as tbllown: 
B(SEQ) = {scq E SEQI seq is a maxinmm 
element in (SEQ; _<c)} • (4) 
The selection procedure consists isl generating 
the various possible sequences, comlmting their 
respective normalized confidence values, and ar- 
bitrarily choosing a member of the set of best 
sequences. It should be noted that not all 
sequences need to be actually generated and 
tested, due to the incorporation of Dijkstra's 
well known "Shortest Path" algorithm (e.g. in 
(Cormen ctal., 1989)). 
4 The Learning Cycle 
Learning the Yes(:aling coefficients in l)erformed 
off-line, and shouht normally take place only 
once, ulsless new training data is asseml)led, or 
new criteria tbr the desirable nystelll l)eh~tvior 
have been tbrmulate(l. Tim learning <;y<:le con- 
sisl, n of incorporating human feedback (training 
set almotation) and finding a set of rescaling 
(:oe\[ticients so as to yield a selection t)ro(:edure 
with optimal or close to optimal accord with the 
human ewfluation. A training set, consisting of 
test dialogues that cover the desirable systens 
functionality, is fed through the system, while 
separately storing the outt)uts produced 1)y the 
various translation modules. These are then 
subject to two phases of mmotation (see sec- 
tion 4.1), resulting in a set of 'best' sequences 
of translated segments tbr each input utterance. 
The next tank is to determine the atsl)ropri - 
ate linear rescaling, that would maximize the 
accord 1)etween the rescaled confidence wflues 
and the 1)references expressed by those 'best' se- 
quences. In order to do that, we first generate a 
large set of ilmqualities as (tescYibed in section 
4.2 below, and then ai)proximate their optimal 
solution, as described in section 4.a. 
845 
4.1 Training Set Annotation 
As mentioned above, evaluating alternative 
translations is a complex task, which some- 
times appears to be difficult even for specially 
trained people. When one alternative seems 
highly appropriate and all the others are clearly 
wrong, a vigilant annotator would normally en- 
counter very little difficulty. But when all op- 
tions fall within the reasonable reahn and differ 
only slightly fl'om one another, or even more 
so, when all options are far from perfect, each 
having its mfiquely combined weaknesses and 
advantages what criterion should be used by 
the annotator to decide which weaknesses are 
more crucial tlmn the others? Our human feed- 
back cycle is twotbld: first, the outputs of the al- 
ternative translations paths are annotated sep- 
arately, so as to enable the calculation of the 
'off-line confidence values' as described below. 
For each dialogue turn, all possible combina- 
tions of translated segments that cover the in- 
put are then generated. For each of those possi- 
ble combinations, an overall off-line confidence 
value is calculated, in a similar way to which 
the 'on-line' confidence is calculated (see sec- 
tion 3), leaving out the rescaling coefficients, 
but keeping the time axis integration. These 
segment combinations are then t)resented to the 
annotators tbr a second round, sorted accord- 
ing to their respective otfline confidence values. 
Tlle annotator is requested at this stage merely 
to select the best segment combination, which 
would normally be one of the first; to appear 
on the list. The first mmotation stage may be 
described as 'theory assisted annotation', and 
the second is its more intuitive complement. %) 
assist the first mmotation rotund we have com- 
piled a set of mmotation criteria, and designed a 
specialized annotation tool for their application. 
These criteria direct the annotator's attention 
to 'essential information items', and rethr to the 
number of such items that have t)een deleted, 
inserted or maintained during the translation. 
Other criteria are the semantic and syntactic 
correctness of the translated utterance as well 
as those of the source utterance. The separate 
annotation of these criteria allows us to express 
the 'off-line confidence' as their weighted linear 
combination. The different weights can be seen 
as implicitly establishing a method of quantify- 
ing translation qnality. One can determine, for 
instance, which is of higher importance -- syn- 
tactical correctness, or the transmission of all 
essential intbrmation items. Using the vague no- 
tion of 'translation qnality' as a single criterion 
would have definitely caused a great divergence 
in personal annotation style and preferences, as 
can be very well exemplified by the case of the 
dialogue act based translation: some people find 
word by word correctness of a translation nmch 
more important than the dialogue act; invari- 
ante, while others argue exactly the opposite 
(Schmitz, 1997),(Schmitz and Quantz, 1995). 
4.2 Generating Inequalities 
Once the best segment sequences for each ut- 
terance have been determined by the completed 
am~otation procedure, a set of inequalities is 
created using the linear rescaling coetficients as 
variables. This is done simply by stating the re- 
quirement that the normalized confidence value 
of the best segment sequence should be better 
than the normalized confidence values of each 
one of the other possible sequences. For each 
utterance with n possible segment sequences, 
this requirement is expressed by (n-1) inequal- 
ities. It is worth mentioning at this point that 
it sometimes occurs during the second annota- 
tion phase, that numerous sequences relating to 
the same utterance are considered 'equally best' 
by the annotator, in such cases, when not all 
sequences are concerned but only a sul)set of 
all possible sequences, we have allowed the an- 
notator to select nnfltiple seqnences as q)est', 
correspondingly multiplying the number of in- 
equalities that are introduced by the utterance 
in question. These multiple sets are known in 
adwmce to be inconsistent, as they in fact for- 
mulate contradictory requirements. Since the 
optinfization procedure attempts to satisfy the 
largest possible subset of inequalities, the logi- 
cal relation between such contradicting sets can 
be seen as disjunction rather than conjunction, 
and they do seem to contribute to the learn- 
ing process, because the different 'equally best' 
sequences are still favored in comparison to all 
other sequences relating to the same utterance. 
The overall resulting set; of inequalities is nor- 
mally very large, and can be expected to be con- 
sistent only in a very idealized world, even in 
the absence of 'equally best' mmotations. The 
inconsistencies reflect many imperfections that 
characterize both the problenl at hand and the 
846 
long way to its solution, inost outstanding of 
which is the fact that the original confidence 
values, as useflfl as they may l)e, are neverthe- 
less far from reilecting the human annotation 
and evaluation results, which are, furthermore, 
not always consistent anlong themselves. The 
rest of the learning process consists in trying to 
satisf~y as many inequalities as possible without 
reaching a contradiction. 
4.3 Optimization Heuristics 
The l)rol)lem of finding the l)est rescaling co- 
eiliciellts reduces itself, under the al)ove inen- 
tioncd presut)t)ositions, to that of fin(ling the 
maxilnal COllsistent sul)set of inequalities within 
a larger, most likely inconsistent, set; of linear in- 
equalities, and solving it. In (Amaldi and Mat- 
tavel\]i, 1997), the problem of extracting close- 
to-lnaxilllUlll consistent subsystelns fi'om an in- 
consistent linear system (MAX CS) is treated 
as part of a strategy for solving the prol)lenl 
of partitioning an inconsistent linear system 
into a lain|real nuIntmr of consistxmt sul)systems 
(MIN PCS). Both t)rol)h',nls are NP-hard, |)uI; 
through a thernlal variation of previous work 
by (Agmon, 1954) and (Motzkin and Schoen- 
berg, 1954), a greedy algorithm is tbrmulated 
t)y (Amaldi and Mattavclli, 1997), which can 
serve as an effective heuristic tbr ol)taining op- 
timal or near to optimal solutions lbr MAX CS. 
hnplementing this algorithm ill the C lmlguagc, 
ellal)led us to comple, te the learning cych', t)y 
tindiug a sol; of coetlicients that maximizes, or 
al. least nearly maximizes, the accord of t;h(; 
rescaled (:onfidence wflues with the judgment 
1)rovided by human aunot;ators. 
5 Additional Information Sources 
llndel)endently of the confidence rescaling pro- 
cess, we have made several attempts to incorpo- 
rate additional latin'mat|on ill order to refine the 
selection procedure. Some of these attempts, 
such as using probabilistic  model in- 
fi)rmation~ or inferring fi'om the logical relation 
between the approximated propositional con- 
te, nts of neighboring utterances (e.g. trying to 
eliminate contra(liction), have not been fruit- 
ful enough to be worth full descril)tion ill the 
present work. The following two sections de- 
scribe two attempts that do seem to be worth 
mentioning in fltrther detail. 
5.1 Dialogue Act Information 
Our experience shows that the translation qual- 
ity that is accomplished by the different mod- 
ules w~ries, among the rest;, according to the 
dialogue act at hand. This seelns to be par- 
ticularly true for syndialog, the dialogue act 
based translation path. Those dialogue acts 
that normally transmit very little propositional 
content, or those that transmit no propositional 
content at all, are normally handled better 
t)y syndialog compared to dialogue acts that 
transmit more information (such as INFORM, 
wlfieh can in principle transmit any proposi- 
tion). The dialogue act recognition algorithm 
used by syndialog does not comt)ute the sin- 
gle most likely dialog act, but rather a probabil- 
ity distril)ution of all possible dialogue acts 1 We 
represent the dialogue act probability distribu- 
tion for a giv(m segment .s by the vector d~t(,~), 
where each component denotes the conditional 
i)rol)al)ility of a certain dialogue act, given the 
segment .s: ( P I ) 
&(.*) = 
The. vectors g and b fl:om section 3 above are re- 
placed by the matrices A and 13 which are sim- 
ply a coneatem~tion of the rest)ective (tbdogue 
a('t v(x'tors• 
lJ,,t X = and 0 = 
The normalized confidence wflue, with. incorpo- 
rated dialogue act information can then be ex- 
pressed as: 
= + • .7). It,ll • 
.sd£'cq 
5.2 Disambiguation Information 
Within the deep translation path, several types 
of underspecitication are used for representing 
ambiguities (Kiissner, 1997), (Kiissner, 1998), 
(Emele and Dorna, 1.998). Whenever an ambi- 
guity has to be resolved in order for the trans- 
lation to succeed, resolution is triggered on de- 
mand (Buschbeck-Wolf, 1997)• Several types 
1For more ilfformation about dialogue, acts in Vca'b- 
mobil, see (Alexandersson ctal., 1997) 
847 
of disambiguation are perfornmd by the context 
module (Koch et al., 2000), which uses various 
knowledge sources in conjunction fbr resolving 
anaphorical and lexical ambiguities. Examples 
for such knowledge sources are world knowl- 
edge, knowledge about the dialogue state, as 
well as various sorts of morphological, syntac- 
tic and semantic information. Since the deep 
translation path is the only one that includes 
contextual disambiguation, its confidence value 
is incremented by the selection module when- 
ever such ambiguities occur. 
6 Quality of Service Parameters 
Translation quality is t)erhaps the most signifi- 
cant Quality of Service (QoS) parameter as far 
as MT systems are concerned. The selection 
module and the learning procedure as described 
above, are indeed ainmd at optimizing this pa- 
rameter. Additionally, we have further exper- 
imented with our selection module in order to 
accommodate for other QoS parameters as well. 
Analogously to QoS in Open Distributed Pro- 
gramming (ODP), we can distinguish t)etween 
the tbllowing main categories: timeliness, vol- 
ume, and reliability. In the timeliness category, 
we refer to the delay from the beginning of the 
acoustic intmt till the begilnfing of the acoustic 
output, which is highly dependent on the sys- 
tem's incrementality. The algorithm described 
so far requires the presence of all translated seg- 
ments within a given dialogue turn, betbre the 
selection itself cast take place. This implies a 
relatively long delay, because the biggest pos- 
sible increment unit, i.e. the whole turn, is 
being used. The maximal increlnentality, and 
therefore the minimal delay, are achieved when 
the first ready segment is being chosen at each 
point. This implies, however, a possible deterio- 
ration in translation quality, and increasing the 
risk that due to segmentation differences across 
modules, no appropriate continuation would be 
found tbr the frst segment that had been cho- 
sen. The latter is referred to as 'loss rate', and 
belongs to the reliability category of QoS di- 
mensions. The trade-off between loss rate and 
incrementality is parameterized by the selec- 
tion module, by selecting a segment as soon as 
n translation modules have delivered segments 
with similar segmentations (1 < n < 4). Within 
the vohnne category, we define the real time fac- 
tor (RTF) as the relation between the overall 
processing time (from the beginning of acous- 
tic input till the end of acoustic output) and 
the overall speaking time (lmginning of acous- 
tic input till the end of acoustic input). In 
order to SUl)port conformance to RTF specifi- 
cation tbr the translation service, the selection 
module supports a QoS signal interface. A QoS 
managenmnt module monitors the runtime be- 
havior of the translation modules, and signals 
the selection process if the estimated R3~F is 
expected to exceed the specification. Upon re- 
ceiving such a signal, the selection module at- 
tempts to complete its output without waiting 
\['or fllrther translated segments. 
7 Conclusion 
We have described certain difficulties that arise 
fl'om the attempt to integrate multiple alterna- 
tive translation paths and to choose their op- 
timal coml)ination into one 'l)est' translatiou. 
Using confidence values that originate fl'om dif- 
ferent translation modules as our basic selec- 
tion criterion, we have introduced a learning 
method which enables us to select in maximal 
accord with decisions taken by human annota- 
tors. Along the way, we have also tackled some 
problematic aspects of translation evaluation as 
such, described some additional sources of in- 
formation that are used by our selection mod- 
ule, and briefly sketched the way in which it; 
supports quality of service specifications. The 
extent to which this module succeeds in creat- 
ing higher quality compound translations is of 
course highly dependent on the appropriate as- 
signment of confidence values, pertbrmed by the 
translation modules themselves. As a rough cri- 
terion tbr evaluating our success, we compared 
the selection module's output to the best re- 
suits achieved by a single translation path. Re- 
cent Verbmobil evaluation results demonstrate 
an improvement of 27.8% achieved by the selec- 
tion module, measured by the number of dia- 
logue turns that were marked 'good' by amm- 
tators who were presented with live alternative 
translations tbr each turn, namely, those deliv- 
ered by the four single paths, and the coml)ound 
translation delivered by the selection module. 
848 

References 

S.Agmon. Th, c rclazatiov, mcth, od for linear in- 
cq'aalitics C;madi:m ,\]ournM of M~tthcmati(:s, 
6:382-392, 19M. 

J.Ale.xmMersson, B.liuschb(~(:k-Wolf, 
T.lhtjin~tmi, M.Kipp, S.Koch, l,kMaier, 
N.1l.cithingcr, B.Schmil;z, M.Siegel. Dialoquc 
Acts in VERBMOBIL-2 Sc<:oud Editio~l, 
\])FKI Saarbriickcn, Universitiil; Sl;ul;l;garl;, 
Technis('he Univ(,rsitSt \]h'rlin, Univ('rsiti~t 
des Sa~MmMcs, Verbmol)il Rot)oft 226, Mai 
\].997. 

E.AmMdi, M.Mattavelli. A combinatorical op- 
timization approach to extract pieccwise tin- 
car structure .\[:rom nonlinear data and a'n ap- 
plication to optical Jto'w scgmantatio'n , TR 
\[)7-12, Corncll Comlml;ational ()t)timiz~tion 
Project, Corn(;ll University, Itlm(:a NY, USA. 

J.Bu(:kow, A.Batlincr, F.Gallwitz, R.Ihfl)cr, 
E.NSth, V.Wm:nke, mM H.Nicmmm. Dove- 
tailing of Acov, stics and Prosody i'n ,5'pov, ta- 
rico'as Speech R, ccognitio',, In Pro(:. int. Conf. 
on St)okcn Languagc Processing, vohmw, 3, 
pages 571-574, Sydlmy, Australia, De,(:cmbcr 
1998. 

B.Buschbc(:k-Wolf. lb:sol,atio',, on Demar~,d. 
1Jnivcrsil;i~t Stuttgarl;. Verl)mol)il l{,et)ort 196. 
May \]997. 

T.(~orlllCll, (Ll~eiscrson, \].\]/,ivcl;. I)~.t'rod'ltction 
to Algorithms MIT \])r(;ss, (Smfl)ri(tgc, Mas- 
sachusetts, \] 98!t. 

M.Emcle, M.Dorna. EJficicnt hnph:mcu, tation 
of a Semantic-based T'lnn.~/~r Approach In 
Proceedings of the 12th Eurot)can Conference 
on Artificial intelligence (ECAI-9(i). August 
1996. 

MZEmclc, M.Dorna. Ambiguity Pre, s+:r,ving Ma- 
chine 7;innslation 'asing l)ackcd Rcprcs'c'uta- 
tions. In Proceedings of tim \]7th Int(;rna- 
l;iomfl Confcrcncc on Comtml;~tionM Linguis- 
tics (COLING-ACL '98), MontreM, Canada. 
August 1998. 

FLlKcdcrking, S.Nirenburg. Thr(',c ltc,,ds arc 
Better than One, ANLP94P, p 95-100, 1994. 
A.Kilger, W.Finklcr. \]ncrcmcntal Generation 
.for Real-Time Applications, DFKI Report; 
RR-95-11, German tl,('sc~rch Center for Ar- 
titiciM Intelligence - DFKI GmbH, 1995. 

M.Kipp, J.Alcxandersson, N.Rcithinger. Un- 
derstanding Spou, tancous Negotiation Dia- 
h).q'uc Proceedings of the, IJCAI Worksho I) 
Knowledge and Reasoning in Practical Dia- 
h)guc Systcllts, Stockhohn, Sweden, August 
1999. 

S.Koch, U.Kfissncr, M.Stcdc. Contc:ctnaI Dis- 
ambig'aation, hi W.Wahlstcr, Ed. Vcrbmobih 
l;b'andations of Spccch, to Speech, 2~'anslation 
Springer Verlag, 2000. 

U.Kiissncr. Applying DL in Automatic Dialogv, e 
Interpreting, Proceedings of the InternaI;ional 
Workshop on Descril)tion Logics - DL-97, pp 
54-58, Gif sur Yvette, France, 1997. 

U.Kiissncr. Description Logic Unplugged Pro- 
ceedings of the, InternationM Workshop on 
Description Logics - DL-98, pp 142-146, 
~lYcnto, Italy, 1998. 

T.S.Motzkin, I.J.Schocnberg. Th, c rclazation 
method for linear inequalities Cmm(tian Jour- 
nal of Mathem~l;ics, 6:393-4()4, 1954. 

F..l.()ch, C.Tillmnnn, lt.Ncy. Improvcd Align- 
mcnt models for Statistical A/htchinc ~J)'au, sla- 
tion, In Proc. of the Joint SIGI)AT Conf. on 
Empirical Methods in Natural Language Pro- 
tossing and Very Large Corpora, Univcrsity 
of Maryland, 1999. 

T.Ruland, C.J.Rut)p, J.Spilker, H.Weber, 
C.Worm. Mak, in 9 th, c Most of Mult@licity: A 
M'ulti-Parscr Multi-Strategy Ar(:h, itcct'ar~: for 
th, c \]?,ob'ast Proccssiu, g of ,5'poke.v, La'n,g'aagc. 
Proceedings of ICSLP 1998. 

M.Schich;n, ,\].Bos, M.l)orna Verbmobil lnl, c,r- 
fat(', ~lhrms (VITs), In W.Wahlsl;er, Ed. 
Ve'rbmobil: I,b',,ndatiou,~" of Spccch, to Spccch, 
'l'ranslation Springer Vcrlag, 2000. 

B.Schmitz. Pragmatikbasicrtcs Masch, inellcs 
Dolmctschcn. \])isscrtation, FB Informatik, 
TU Berlin, 1997. 

B.S(:hmitz, J.J.Quantz. Diah)guc Acts in Auto- 
matic Dialoquc Interpreting in Proceedings of 
the Sixth inl;crll~l;ion~l Conference OlI rlThco- 
rcticM and McthodologicM Issues in Machine 
'lYanslation (TMI-95), Lcuven, 1995. 

W.Wahlster, Ed. Vcrbmobih ~bundations of 
Speech, to Speech, ~}'anslation Springer Vcrlag, 
2000. 

C.Worm, C.J.l:l,ut)t). Towards Robust Uudcr- 
standing of Spccch by Combination of Partial 
Analyses Proceedings of ECAI 1998 
