An Experiment On Incremental Analysis 
Using Robust Parsing Techniques 
Kilian Foth and Wolfgang Menzel and Horia F. Pop and Ingo Schr6der 
foth I menzel I hfpop I schroeder@nats.infonnatik.mfi-hamburg.de 
Fachbereich Informatik, Universitgt Hamburg 
Vogt-K611n-Strage 30, 22527 Hamburg, Germany 
Abstract 
The results of an experiment are presented in which 
an approach for robust parsing has been applied in- 
crementally. They confirm that due to the robust na- 
ture of the underlying technology an arbitrary pre- 
fix of a sentence can be analysed into an interme- 
diate structural description which is able to direct 
the further analysis with a high degree of reliability. 
Most notably, this result can be achieved without 
adapting the gramrnar or the parsing algorithms to 
the case of increinental processing. The resulting 
incremental parsing procedure is significantly faster 
if compared to a non-incremental best-first search. 
Additionally it turns out that longer sentences ben- 
efit most from this acceleration. 
1 Introduction 
Natural  utterances usually unfold over 
time, i. e., both listening and reading are carried 
out in an incremental left-to-right manner. Model- 
ing a similar type of behaviour in computer-based 
solutions is a challenging aim particularly interest- 
ing task for a number of quite different reasons that 
are most relevant in the context of spoken  
systems: 
• Without any external signals about the end 
of an utterance, incremental analysis is the 
only means to segment the incoming stream of 
speech input. 
• An incremental analysis mode provides for a 
lnore natural (mixed-initiative) dialogue be- 
haviour because partial results are already avail- 
able well before the end of an utterance. 
• Parsing may already take place in concurrency 
to sentence production. Therefore the speaking 
time becomes available as computing time. 
• Dynmnic expectations about the upcoming 
parts of the utterance might be derived right in 
time to provide guiding hints for other process- 
ing components, e. g., predictions about likely 
word forms for a speech recognizer (Hauenstein 
and Weber, 1994). 
In principle, two alternative strategies can be pur- 
sued when designing all incremental parsing proce- 
dure: 
1. To keel) open all necessary structural hypothe- 
ses required to accomodate every possible con- 
tinuation of an utterance. This is tile strategy 
usually adopted in an incremental chart parser 
(WirSn, 1992). 
2. To commit to one or a limited number of inter- 
pretations where 
(a) either this commitment is made rather 
early and a mechanisnl for partial re- 
analysis is provided (Lombardo, 1992) or 
(b) it is delayed until sufficient information is 
eventually available to take an ultimate de- 
cision (Marcus, 1987). 
The apparent efficiency of human  un- 
derstanding is usually attributed to an early com- 
mitment strategy. 
Our approach, in fact, represents an attempt to 
combine these two strategies: On the one hand, it 
keeps many of the available building blocks for the 
initial part of an utterance and passes an ('updated) 
search space to the following processing step. Ad- 
ditionally, the optimal structural description for the 
data already available is determined. This not only 
makes an actual interpretation (together with expec- 
tations for possible continuations) available to sub- 
sequent processing components, but also opens up 
the possibility to use this information to effectively 
constrain the set of new structural hypotheses. 
Determining the optimal interpretation for a yet 
incomplete sentence, however, requires a parsing ap- 
proach robust enough to analyse an arbitrary sen- 
tencc prefix into a meaningful structure. Therefore 
two closely related questions need to be raised: 
1. Can the necessary degree of robustness be 
achieved? 
2. Is the information contained in the currently 
optimal structure useful to guide the subsequent 
analysis? 
1026 
To answer these questions and to estimate the po- 
tential for search space reductions against a 1)ossible 
loss of accuracy, a series of experiments has been 
conducted. Sections 2 and 3 introduce and motivate 
the framework of the exl)eriments. Section 4 de- 
scribes a number of heuristics for re-use of previous 
solutions and Section 5 1)resents the results. 
2 Robust Parsing in a Dependency 
Framework 
Our grammar models utterances as depertdeTzcy 
t ryes, which consist of pairs of words so that one 
depends directly on the other. This subordination 
relation can be qualified by a label (e. g. to distin- 
guish conq)lements fl:om modifiers). Since each word 
cml only depend on one other word, a labeled tree 
is formed, usually with the finite verb as its root. 
The decision on which stru(:ture to 1)ostulate for 
an utterance is guided 1)y explicit constrai~,ts, which 
are rel)resented as mfiversally quantified logical for- 
mulas about features of word tbrms and l)artial trees. 
t~r instance, one constraint might t)ostulate that a 
relation labeled as 'Subject' can only occur between 
a noun and a finite verb to its right, or that two dif- 
ferent del)endencies of the same verb may not both 
be labeled 'Sul)ject'. For efficiency reasons, these 
formulas may constrain individual del)endency edges 
or pairs of edges only. The al)plication of constraints 
Call I)egin as soou as the first word of an utterance 
is read; no global information about the utterance is 
required for analysis of its beginning. 
Since natm:al  inl)ut will often exhibit ir- 
regularities such as restarts, repairs, hesitations and 
other gr~tmmatical errors, individual errors should 
not make further analysis impossible. Instead, a ro- 
bust 1)arser should continue to build a structure tbr 
the utterance. Ideally, this structure should be close 
to that of a similar, but grammatical utterance. 
This goal is attained by annotating the constraints 
that constitute the grammar with scores ranging 
from 0 to 1. A structure that violates one or more 
constraints is annotated with the product of the col 
responding scores, and the structure with the highest 
combined score is defined as the solution of the pars- 
lug problem. In general, the higher the scores of the 
constraints, the more irregular constructions can 1)e 
analysed. Parsing an utterance with mmotated or 
soft constraints thus mnounts to multi-dimensional 
optimization. Both complete and heuristic search 
methods can be eml)loyed to solve such a t)roblem. 
Our robust al)proach also provides m~ easy way 
to implement partial parsing. If necessary, e. g., an 
isolated noun labeled as 'Subject' may form the root 
of a del)endency tree, although this would violate 
the first constraint lnentioned above. If a finite verl) 
is available, however, subordinating the noun under 
the verb will avoid the error and thus produce a 
better structure. This capability is crucial for the 
analysis of incomplete utterances. 
Different lcvcls of analysis can be defined to model 
syntactic as well as semantic structures. A depel> 
dency tree is constructed for each of these levels. 
Since constraints can relate the edges in parallel de- 
pendency trees to each other, having several trees 
contributes to the robustness of the approach. Al- 
together, the gramlnar used in the experiments de- 
scribed comprises \] 2 levels of analysis and 490 con- 
straints (SchrSder el, al., 2000). 
3 Prefix Parsing with Weighted 
Constraints 
In general, dependency analysis is well-suited for 
incremental analysis. Since subordinations always 
concern two words rather than full constituents, each 
word can be integrated into the analysis as soon as it 
is read, although not necessarily in the ol)timal way. 
Also, the 1)re-colnl)uted dependency links can easily 
1)e re-used in subsequent iterations. Therefore, de- 
1)endency grammar allows a fine-grained incremental 
analysis (boxnbardo, 1992). 
mr c/"\, r2 "- mod 
o" • \o • 
daal; lassen sie uns doch 
I hen lat you us 
z" c1.,. I 
j" 
~0 
mod// /. 
rloch einen Ten'nil; 
<pa~t> yet a meeting 
(a) 
-\ 
c "~ mod 
\\ ~o 0 0 / ~ \', 
dana lassen sie uns doch 
Then let you us <part> 
Let's appoint yet another meeting then. 
O" 
cJ -" : 
i IB" 
,,,oa ~ ~ i (b) 
o~ ~ 
noch einoll Terrain ausmachen 
yel a meeting appoint 
lPigure 1: An example for a prefix analysis 
When assigning a det)endency structure to incom- 
plete utterances, the problem arises how to analyse 
words whose governors or complements still lie be- 
yond the time horizon. Two distinct alternatives are 
1)ossible: 
1. The parser can establish a dependency between 
tilt word and a special node representing a im- 
tative word that is assmned to follow in the re- 
maining input. This explicitly models the ex- 
pectations that would be raised by the prefix. 
llowever, unifying new words with these under- 
specified nodes is difficult, particularly when 
1027 
multiple words have been conjectured. Also, 
many constraints cannot be meaningfully ap- 
plied to words with unknown features. 
2. An incomplete prefix can be analyzed directly 
if a grammar is robust enough to allow par- 
tial parsing as discussed in the previous sec- 
tion: If the constraint that forbids multiple 
trees receives a severe but non-zero penalty, 
missing governors or complements are accept- 
able as long as no better structure is possible. 
Experiments in prefix parsing using a dependency 
grammar of German have shown that even complex 
utterances with nested subclauses can be analysed 
in the second way. Figure la provides an example 
of this: Because the infinitive verb 'ausmachen' is 
not yet visible, its coinplement ~Terinin' is analysed 
as an isolated subtree, and the main verb 'lassen' is 
lacking a comI)lement. After the missing verb has 
been read, two additional dependency edges suffice 
to build the correct structure from the partial parse. 
This method allows direct comparison between in- 
cremental and non-incremental parser runs, since 
both methods use tim same grammar. Therefore, 
we will follow up on the second alternative only and 
construct extended structures guided by the struc- 
tures of prefixes, without explicitly modeling missing 
words. 
4 Re-Use of Partial Results 
While a prefix analysis can produce partial parses 
and diagnoses, so far this inforlnation has not been 
used in subsequent iterations. In fact, after a new 
word has been read, another search is conducted on 
all words already available. To reduce this duplica- 
tion of work, we wish to narrow down the problem 
space for these words. Therefore, at each iteration, 
the set of hypotheses has to be updated: 
• By deciding which old dependency hypotheses 
should be kept. 
• By deciding which new dependency hypotheses 
should be added to the search space in order to 
accomodate the incoming word. 
For that purpose, several heuristics have been de- 
vised, based on the following principles: 
Prediction strength. Restrict the search space as 
much as possible, while maintaining correct- 
ness. 
Economy. Keep as nmch of the previous structure 
as possible. 
Rightmost attachment. Attach the incoming 
word to the most recent words. 
The heuristics are presented here in increasing or- 
der of the size of the problem space they produce: 
A. Keep all dependency edges from the previous 
optimal solution. Add all dependency edges 
where the incoming word modifies, or is modi- 
fied by, another word. 
B. As A, but also keep all links that differ h'om the 
previous optimal solution only in their lexical 
readings. 
C. As B, but also keep all links that differ fl'om the 
previous optimal solution only in the subordi- 
nation of its last word. 
D. As C, but also keep all links that differ from the 
previous optimal solution only in the subordi- 
nation of all the words lying on the path h'om 
the last word to the root of the solution tree. 
E. As D, but for all trees in the previous solution. 
5 Results 
In order to evaluate the potential of the heuristics 
described above, we have conducted a series of ex- 
periments using a grammar that was designed for 
non-incremental, robust parsing. We tested the in- 
crelnental against a non-incremental parser using 
222 utterances taken from the VERBMOBIL do- 
main (Wahlster, 1993). 
V:\] -o -.\[=\] 
1 r; 
A B C D E 
Heuristics 
Figure 2: Solution quality and processing time for 
different heuristics 
Figure 2 compares the five heuristics with respect 
to tile following criteria: 1 
Accuracy. The accuracy (gray bar) describes how 
many edges of the solutions are correct. 
correct edges 
accuracy = ~ edges found 
tNote that the heuristics provide at most one solution and 
may fail to find any solution. 
1028 
Weak recall. We base our recall measure - given as 
the black bar - on the number of solutions found 
non-incrementally (which is less than 100%) be- 
cause we want focus on tile impact of our heuris- 
tics, not the coverage of tile grammar. 
weak recall = @ correct edges 
@ edges fouud non-incrementally 
Relative run-time. The run-time required by the 
incremental procedure as a percentage of the 
time required by the non-iucremental search al- 
gorithm is given as the white bar. 
The difference between tile gray and the black bar 
is due to errors of the heuristic method, i. e., ei- 
ther because of its incal)ability to find the correct 
subordination or due to excessive resource demands 
(which lead to process abortion). 
2Nvo observations can be made: First, all but 
tt,e last heuristics need less time than the non- 
incremental algorithm to complete while maintain- 
ing a relative high degree of quality. Second, the 
more elaborate the heuristics are, the longer they 
need to run (as expected) and the better are the re- 
sults for the accuracy measure. However, the lmuris- 
tics D and E could not complete to parse all sen- 
tences because in some cases a pre-defined time limit 
was exceeded; this leads to the observed decrease 
in weak recall when compared to heuristics C. As 
expected, a trade-off between computing time and 
quality can be found. Overall, heuristics C seems to 
be a good choice because it achieves all accuracy of 
up to 93.7% in only one fifth of the run-time. 
250%: 
200% 
t50% 
~00% 
50% 
Figure 
Figure 
heuristics 
~ Time for incremen- 
tal compared to non- 
incremental method 
D Absolute time for in- 
cremental (16...20 set 
to 100%) 
• The gray bar presents tile normalized time with 
the time for sentence length between 16 and 20 
set to 100%. 
Tile results show that the speedup observed in 
Figure 2 is not evenly distributed. While the incre- 
mental analysis of the short sentences takes longer 
(2.5 times slower) than the non-incremental algo- 
rithm, the opposite is true for longer sentences (10 
times faster). However, this is welcome behavior: 
The incremental procedure takes longer only in those 
cases that are solved very fast anyway; tim problem- 
atic cases are parsed more quickly. This behavior is 
a first hint that the incremental analysis with re-use 
of partial results is a step that alleviates the combi- 
natorial explosion of resource demands. 
-° ~ ~ 
m ~ cd 
100%" c-i ,, 
80%. 
60%" 
40%' 
20% 
O% 1...5 6...10 11...15 16...20 overall 
Sentence Length 
3: Processing time vs. sentence length 
3 coral)ares the time requirements of 
C for different sentence lengths. 
• The relative run-time (as in Figure 2) is given 
as the wlfite bar. 
0%, 
1...5 6...10 11...15 16...20 overall 
Sentence Length 
Figure 4: Accuracy vs. sentence length (colors have 
tile same meaning as ill Figure 2) 
Finally, Figure 4 compares the quality resulting 
from heuristics C for different sentence lengths. It 
turns out that, although a slight decrease is observ- 
able, the accuracy is relatively independent of sen- 
tence length. 
I I 
• ~ r-q~ ~ I ~.1 I ~ ~ 6 Conclusions co c,~ 
m 6 ~ An apl)roach to the incremental parsing of natural 
 utterances has been presented, which is 
based on tlle idea to use robust parsing techniques 
to deal with incomplete sentences. It determines a 
structural description for arbitrary sentence prefixes 
by searching for the optimal combination of local 
hypotheses. This search is conducted in a problem 
space which is repeatedly narrowed down according 
to the optimal solution found in tile preceding step 
of analysis. 
1029 
The results available so far confirm the initial ex- 
pectation that the grammar used is robust enough 
to reliably carry out such a prefix analysis, al- 
though it has originally been developed for the non- 
incremental case. The optimal structure as deter- 
mined by the parser obviously contains relevant in- 
formation about the sentence prefix, so that even 
very simple and cheap heuristics can achieve a con- 
siderable level of accuracy. Therefore, large parts of 
the search space can be excluded fi'oln repeated re- 
analysis, which eventually makes it even faster than 
its non-incremental counterpart. Most importantly, 
the observed speedup grows with the length of the 
utterance. 
On the other hand, none of the used structure- 
based heuristics produces a significant iml)rovement 
of quality even if a large amount of computational 
resources is spent. Quite a number of cases can 
be identified where even the most expensive of our 
heuristics is not strong enough, e. g., the German 
sentence with a topicalized direct object: 
DieNOM,ACe Frau sieht derNOM Mama. 
The woman sees tile man. 
The woman, the man sees. 
Here, when analysing the subsentence die Frau 
sieht, the parser will wrongly consider die Frau as 
the subject, because it appears to have the right 
case and there is a clear preference to do so. Later, 
when the next word comes in, there is no way to 
allow for dic Frau to change its structural interpre- 
tation, because this is not licensed by any of the 
given heuristics. 
Therefore, substantially more i)roblenl-oriellted 
heuristics are required, which should take into ac- 
count not only the ol)timal structure, but also the 
conflicts caused by it. Using a weak but cheap 
heuristics, a fast al)proximation of the optimal struc- 
ture can be obtained within a very restricted search 
space, and then refined by subsequent structural 
transformations (Foth et al., 2000). To a certain 
degree this resembles the idea of applying reason 
maintenance techniques for conflict resolution in in- 
cremental parsing (Wir6n, 1990). In deciding which 
strategy is good enough to find the necessary first 
approximation the results of this paper might play a 
crucial role, since the I)ossible contribution of indi- 
vidual heuristics in such all extended fi'amework can 
be precisely estimated. 
Acknowledgements 
This research has been partly fimded by the German 
Research Foundation "Deutsche Forschungsgemein- 
schaft" under grant no. Me 1472/1-2. 

References 

Kilian Foth, Wolfgang Menzel, and Ingo SchrSder. 
2000. A transformation-based parsing technique 
with anytime property. In Procecdings of the 
International Workshop on Parsing Technologies 
(IWPT-2000), pages 89-100, Trento, Italy. 

Andreas Hauenstein and Hans Weber. 1994. An in- 
vestigation of tightly coupled time synchronous 
speech  interfaces using a unification 
grammar. In Proceedings of the i2th National 
Conference on Artificial Intelligence: Workshop 
on the Integration of Natural Language and Speech 
Processing, pages 42-49, Seattle, Washington. 

Johannes Heinecke, J/irgen Kunze, Wolfgang Men- 
zel, and Ingo SchrSder. 1998. Eliminative pars- 
ing with graded constraints. In Proceedings of 
the Joint Conference COLING/ACL-98, Mon- 
trial, Canada. 

Vincenzo Lombardo. 1992. Incremental dependency 
parsing. In P~vceedings of the Annual Meeting of 
the ACL, Delaware, Newark, USA. 

Mitchell P. Marcus. 1987. Deterministic pars- 
ing and description theory. In Peter Whitclock, 
Mary McGee Wood, Harold Seiners, Rod John- 
son, and Paul Bennett, editors, Linguistic Theory 
and Computer Applications', pages 69-112. Aca- 
demic Press, London, England. 

Wolfgang Menzel and ingo SchrSder. 1998. Decision 
procedures for dependency parsing using graded 
constraints. In Sylvain Kahane and Alain Pol- 
gu~re, editors, Proceedings of the Joint Confer- 
ence COLING/ACL-98 Workshop: Processing of 
Dependency-based Grammars, pages 78-87, Mon- 
trSal, Canada. 

Ingo SchrSder, Wolfgang Menzel, Kilian Foth, 
and Michael Schulz. 2000. Modeling depen- 
dency grammar with restricted constraints. In- 
ternational Journal J?'aitemcnt Automatique des 
Langues: Grammaires de d@endance, 41(1). 

Wolfgang Wahlster. 1993. Verbmobih Translation 
of face-to-face dialogs. In Proceedings of the 3rd 
European Conference on @eech Communication 
and Technology, pages 29-38, Berlin, Germany. 

Itans Weber. 1995. LR-inkrementelles proba- 
bilistisches Chartparsing von Worthypothesen- 
graphen mit Unifikationsgrammatiken: Eine enge 
Kopplung von Suche und Analyse. Verbmobil- 
Report 52, UniversitAt Erlangen-Niirnberg. 

Mats Wir6n. 1992. Studies in Incremental Natural- 
Language Analysis. Ph.D. tlmsis, Department of 
Computer and Information Science, LinkSping 
University, LinkSping, Sweden. 

Mats WirSn. 1990. Incremental parsing and reason 
maintenance. In Proceedings of the i3th Interna- 
tional Conference on Computational Linguistics 
(COLING-90), pages 287-292, Helsinki, Finland. 
