I)YNAMICS, DEPENI)ENCY GRAMMAR AND INCREMENTAL INTERPRETATION* 
DAVID MILWARI) 
Centre for Cognitive Science, University of Edinburgh 
2, lhn:cleuch Place, Edinburgh, ElI8 9LW, Scotland 
davidmC~cogsci.ed, ac.uk 
Abstract 
The paper describes two equiwtlent grammatical for- 
malisnLs. The first is a lexicalised version of depen 
dency grammar, and this can be nsed to provide 
tree-structured analyses of sentences (though some- 
what tlatter than those usually provided by phra.se 
structure grammars). The second is a new forrnal 
ism, 'Dynamic Dependency Gramniar', which uses 
axioms and deduction rules to provide analyses of 
sentences in terms of transitioos between states. 
A reformulation of dependency grammar usiug 
state transitions is of interest on several grounds. 
Firstly, it can be used to show that incremental in- 
terpretation is possible without requiring notions of 
overlapping, or flexible constituency (as ill some ver- 
sions of categorial grammar), and without destroy 
ing a trasmparent link between syntax and seman- 
tics. Secondly, the reformulation provides a level of 
description which can act as an intermediate stage 
between the original grammar and a parsing algo- 
rithm. Thirdly, it is possible to extend the relbrnm 
lated grammars with further axioii~s and deduction 
rules to provide coverage of syntactic constructions 
such as coortlination which are tlitficult to encode 
lexically. 
1 Dynamics 
Dynantics can roughly be described ms the study (ff 
systems which consist of a set of states (cognitive, 
physical etc.) and a family of binary lmnsil~on re- 
lalionships between states, corresponding to actions 
which can be performed to change from one state to 
another (van Benthem, 1990). 
This paper introduces a notion of dynamic yram.. 
Slat, where each word ill a sentence is treated a~s an 
action which ha.s the potential to produce a change in 
state, and each state encodes (in some form) the syn- 
tactic or semantic dependencies of the words which 
|lave been absorbed so far. There is no requirement 
for tile number of states to he finite. (ln fact, since 
dependency grammar allows centre embedding of ar- 
bitrary depth, the corresponding dynamic grammar 
provides an unlimited number of states). 
Dynamic grammars are specified using very sim- 
ple logics, and a sentence is accepted ,~s grammatical 
if and only if there is some proof that it perforn~s 
a transition between some suitable initial and final 
*This resen.rch w~.s nUplmrted by ml SERC research 
fellowship. 
states, It is worth noting at this early stage that dy- 
namic grammars are not lexicalised rehashes of Aug- 
mented Transition Networks (Woods, 1973). A'I'Ns 
use a finite number of states combined with a re- 
cursion mechanism, and act ea'~entially in the same 
way ms a top down parser. They are not particularly 
suited to increment,'d interpretation. 
To get an idea of how logics (instead of the more 
usual algebra.s) can be used to specify dynamic sys- 
tenLs in general, it is worth considering a reformula- 
tion of the {bllowing finite state machine (FSM): 
h 
0 l 2 :1 
This accepts .'~ grammatical any string which maps 
from the initial state, 0, to the final state, 3 (i.c. 
strings of the form: nb*eb). The FSM cart be refor- 
mulated using a logic where the notation, 
StateO Str Statel 
is used to state that the string, Str, perfornm a 
transition from StateO to Statel. The axioms (or 
atomic proofs) in tile logic are provided by the tran- 
sitions peribrnted by the individual letters. Thus the 
following are mssumed, t 
0 "a" t 1 "b" I 2 "b" 3 1 "c" 2 
The transitions given by the single letter strings are 
put together using a deduction rule, Sequencing, 2 
which states that, provided there is a proof that 
String, takes us from some state, So, to a state 
S t and a proof that Stringb takes us from St to $2, 
then there is a proof that the concatenation of the 
strings takes us from S0 to $2. The rule puts to 
aether strings of letters if the final state reached by 
tile first string matches an initial state for tile secontl 
string. For example, the rule may be instantiated as: 
A string is grammatical according to the logic if and 
only if it is possible to construct a proof of the state- 
ment 0 Str 3 using the axioms and the Sequencing 
Rule. For example, the string "abbcb" performs the 
transitions, 
l "a" is a ~trin~ coxudsting of the single letter, a. 
2Notation: capital lettet~ will be used to denote variables 
throughout this paper. 'a' will be used to denote eoncatena~ 
tion. For example, if Stringa = "kl" mad String b = "atilt", 
then StrlngaeStrlngb = ukltllllll. 
ACRES DE C()LING. 92, NAN-rES. 23-28 AOt'rr 1992 l 0 9 5 l)Roe, ov COLING-92, NA~I'ES AUG. 23-28, 1992 
and has the following proof, amongst others: 
1 "b" l 1 "b" 1 
1 "bb" 1 1 "c" 2 
0 "a" 1 1 "bbc" 2 
0 "abbc" 2 2 "b" 3 
0 "abbcb" 3 
Each leaf of the tree is an axiom, and the subproofs 
are put together using instantiations of the Sequenc- 
ing Rule. 
2 Lexiealised Dependency Grammar 
"lYaditional dependency grammar is not concerned 
with constituent structure, but with links between 
individual words. For example, an analysis of the 
sentence John thought Mary showed Ben ~o Sue 
might be represented as follows: 
John thought Mary showed Ben to Sue 
The word thought is the head of the whole sentence, 
and it has two dependents, John and showed, showed 
is the head of the embedded sentence, with three de- 
pendents, Mary, Ben and to. A dependency graph is 
said to respect adjacency if no word is separated from 
its head except by its own dependents, or by another 
dependent of the same head and its dependents (i.e. 
there are no crossed links). Adjacency is a reason- 
ably standard restriction, and has been proposed as 
a universal principle e.g. by Hudson (1988). 
Given adjacency, it is possible to extract bracketed 
strings (mid hence a notion of constituent structure) 
by grouping together each head with its dependents 
(and the dependents of its dependents). For exam- 
ple, the sentence above gets the bracketing: 
\[John thought \[Mary showed Ben \[to Sue\]\]\] 
A noun phrase can be thought of as a noun plus all 
its dependents, a sentence as a verb plus all its de- 
pendents. 
In this paper we will assume adjacency, and, for 
simplicity, that dependents are fixed in their order 
relative to the head and to each other. Depen- 
dency grammars adopting these assumptions were 
formalised by Gaifman (Hays, 1964). Lexicalisation 
is relatively trivial given this formalisation, and the 
work on embedded dependency grammar within cat- 
egorial grammar (Barry and Picketing, 1990). 
Lexiealised Dependency Grammar (LDG) treats 
each head as aflmction. For example, the head 
lhought is treated as a function with two arguments, 
a noun phrase and a sentence. Constituents are 
formed by combining functions with all their argu- 
ments. The example above gets the following brack- 
eting and tree structure: 
\[John thought \[Mary showed Ben \[to Sue\]\]\] 
s 
. ..... I ~'-~ 
llnp) // 
L ~ls~ J , 
ilnpl \ 
Lr(np, pp) 
PP 1 np 10 
Lrtnp) 
The tree structure is particularly flat, with all ar- 
guments of a function appearing at tile same level 
(this contrasts with standard phrase structure anal- 
yses where the subject of showed would appear at a 
different level from its objects). 
Lexical categories are feature structures with three 
main features, a base type (the type of the con- 
stituent formed by combining the lexical item with 
its arguments), a list of arguments which must ap- 
pear to the left, and a list of arguments which must 
appear to the right. The arguments at the top of the 
lists must appear closest to the functor. For example, 
showed has the lexical entry, 
I 1 showed : llnp) 
L=(-p, ppl J 
and can combine with an np on its left and with an 
np and then a pp on its right to form a sentence. 
When left and right argument lists are empty, cat- 
egories are said to be saturated, and may be written 
as their base type i.e. \[X)\] isidenticaltoX. 
L~clj 
A requirement inherited from dependency grammar 
is for arguments to be saturated categories, a LDGs 
will be specified more formally in Section 4. 
It is worth outlining the differences between the 
categories in LDG and those in a directed categorial 
grammar. Firstly, in LDG there is no specification 
of whether arguments to the right or to the left of 
the functor should be combined with first. Thus, 
the category, I(Y) , maps to both (X\Y)/Z and 
LrlZ~ J (X/Z)\Y, 4 Seeondly, arguments in LDG must be 
saturated, so there can be no functions of functions. 5 
aIn dependency granunar it is not ponaible to specify that 
a head requirea a dependent with only some, but not all of its 
dependenta. 
4So called 'Steedrn~n' notation. 
5An extended form of LDG which allows ungaturated argu- 
ACRES DE COLING-92. NANTES, 23-28 AOITI" 1992 1 096 PROC. OF COLING-92, NANTES, AUO. 23-28, 1992 
3 Dynamic Dependency Grammar 
Lexicali~d Dependency Grammar can be reformu- 
lated as the dynamic grammar, Dynamic Depen- 
dency Grammar (DDG). Each state in DDG encodes 
the syntactic context, and is labelled by the typc of 
the string absorbed so far. For example, a possible 
set of transitions for the string of words "Sue saw 
Ben" is as follows: 
' Ii 1 S Sue \]J'etl L~lsl j ~l ~,up, , L~-Inplj 
The state after absorbing "Sue saw" is of type sen- 
tence mi~ing a noun phrase, and that after absorb- 
lug "Sue saw Ben" is of type sentence. 
States are labelled by complex categories which arc 
similar to tile lexical categories of LDG, but without 
the restriction that arguments must be saturated (for 
example, the state after absorbing "Sue" has an un- 
saturated argument on its right list). A string of 
words, Str, is grammatical provided tile following 
statement can be proven: 
i 0 Str s 
\[r(s} 
The initial state is labelled with an identity type i.e. 
a sentence missing a sentence. This cat, be thought 
of as a context in whic}~ a sentence is expected, or as 
a suitable type for a string of words of length zero. 
The final state is just of type sentence. 
DDG is specified nsing a logic consisting of a set of 
axioms and a deduction rule. The logic is similar, but 
more general, than that used in Axiomatic Grammar 
(Milward, 1990)f The deduction rule is again called 
Sequencing. The rule is identical in form to the Se- 
quencing Rule used in the reformulation of the FSM, 
though here it puts together strings of words rather 
tiian strings of letters. The rule is as follows, ~ 
Co String~ C,~ Ct String, C~ 
Co String.oString b C~ 
and is restricted to non-empty strings, s 
menks h&s been developed, and this also e2tn be reformulated 
as a dynamic ~,m*mma" (Milward, 1992). 
6Axiomatic Grmnm~r is a particular dynamic grammar de- 
signed for English, which take~ relationslfips between states 
tm a primary phenomenon, to be justified solely by linguistic 
data (rather thmt by an existing formalism such as dependency 
granmlar). 
ZHere 'o' concatenates strings of words e.g. 
"John"o"~leepa" = "John sleeps". 
SThis re~trictlon is not actually necessary as far as the 
equivalence between LDGs and DDGa is concerned. However 
its inclusion makes it trivial to show certain fontud properties 
of DDGs, such a.~ termination of proofs. 
The set of axioms is infinite since we need to 
consider transitions between an arbitrary number of 
categories. 9 The set can be described using just two 
axiom schemata, Prediction and Application. Pre- 
diction is given below, but is best understood by 
considering various instantiations) ° 
i0 iO 
I t( IL,L' ),It' rR,( IIX),L' ,R' L~O J L ~o 
LrRj 
Prediction is usually used when the category of tile 
word encomitered does not match the category ex- 
pected by the current state. Consider the following 
instantiation: 
i 0 "Sue" s where Sue:np 
\[rls) r( llnp) ) 
LrO 
The current state expects a seutence and encounters 
a noun phrase with \]exical entry Sue:rip. The result- 
ing state expects a sentence missing a noun phrase 
on its left e.g. a verb phrase. 
Application gets its name from its similarity to 
function application (though it actually plays the 
role of both application and composition). The 
schema is ~ follows: I 
'\[1 I 1 I:,l i0 s "W" 10 where W: r( XIL ).R' \[rlt.R'J krR\] 
rOJ 
An example instantiation is when a noun phrase i~ 
both expected and encountered e.g. \[sl 
l 0 "Ben" s where Ben:np 
L rl up, j 
Given a word and a particular current state, the 
only non-determinism in forming a resnlting state is 
due to lexical ambiguity or from a choice between us- 
ing Prediction or Application (Prediction is possible 
whenever Application is). Non-determinism is gen- 
°An infinite nmnber of distinguishable stat~ is required to 
deal with centre embedding. 
X°L, L ~, R dad R' are lists of categories. '#' COhCatermtes 
lists e.g. (k,l} • (,non} = (k,l,m,n). 
AC1ES DE COTING-92, Nnl, rn~.s, 2.3~28 Ao~' 1992 1 0 9 7 P~toc. OF COL1NG-92, N^NaXS, AUO. 23-28, 1992 
erally kept low due to states being labelled by types 
as opposed to explicit tree structures. This is easiest 
to illustrate using a verb final language. Consider a 
pseudo English where the strings, "Ben Sue saw" and 
"Ben Sue sleeps believes" are acceptable sentences, 
and have the following LDG analyses: 
l l(np, L r° np} 
s 
/\ 
l(np) 
Lr(I J 
Despite the differences in the LDG tree structures, 
the initial fragment of each sentence, "Ben Sue", can 
..be treated identically by the corresponding DDG. 
The proof of the transition performed by the string 
"Ben Sue" involves two applications of Prediction 
put together using the Sequencing Rule. The transi- 
tions are as follows: 
11) ~Z'~n s ~t s Lr(s)j 
rl l(np) ) r( l(np, np) ) 
L r(l L L~ 
The transitions for the two sentences diverge when 
we consider the words saw and sleeps. In the former 
case, Application is used, in the latter, Prediction 
then Application. 
Efficient parsing algorithms can be based upon 
DDGs due to this relative lack of non-determinism in 
choosing between states. H The simplest algorithm 
is merely to non-deterministically apply Prediction 
and Application to the initial category. Derivations 
of algorithms from more complex logics, and the use 
of normalised proofs and equivalence classes of proofs 
are described in Milward (1991). 
4 LDGs --+ DDGs 
An LDG can he specified more formally as follows: 
1. A finite set of base types To, .. "In (such as s, 
up, and pp) 
llDetermlnism can also be increased by restricting the ax- 
ioms according to the properties of a particular lexicon. For 
example, there is no point predicting categories missing two 
noun phrases to the left when pansing English. 
2. An infinite set of lexical categories of the form, 
XL1 where X is a base type, and L and It are rR\] 
lists of base types. When L and R are empty, a 
category is identical to its base type, X 
3. A finite lexicon, L, which assigns lexical cate- 
gories to words 
4. A distinguished base type, To. A string is gram- 
matieal iff it has the category, To 
5. A combination rule stating that, F \] 
if W has category, 1( Ti,.., TI) 
Jr( 7,+a ..... Ti+i) 
and String1 has category T1, String2 has cate- 
gory 7~ etc. 
then the string formed by concatenating 
String1, .. ,String~, "W', Stringi+l, .. ,String/+j 
has category X 
corresponding DDG is as follows: 
.... 
L~Rj where X is a base type, and L and Yt are lists of 
categories 
2. Two axiom schemata, Application and Predic- 
tion 
3. The lexicon, L (as above) 
4. One deduction rule, Sequencing I \] 
5. A distinguished pair of categories, ll) , 7~ 
L~:(nlJ 
where To is as above. A string, Str, is grammat- 
ical iff it is possible to prove: \[ 
1'7(~°) \] Str 7~ 
r(To) j 
A proof that any DDG is strongly equivalent to its 
corresponding LDG is given by Milward (1992). The 
proof is split into a soundness proof (that a DDG 
accepts the same strings of words and assigns them 
corresponding analysesl2), and a completeness proof 
(that a DDG accepts whatever strings are accepted 
by the corresponding LDG). 
The 
1. 
5 Incremental Interpretation 
It is possible to augment each state with a semantic 
type and a semantic value. Adopting a 'standard' A- 
calculus semantics (c.f. Dowty et al, 1981) we obtain 
the following transitions for the string "Sue saw": 
12For this purpose, it is convenient to treat an analysis in a 
DDG as the traasition~ performed by each word. Each anal- 
ysis is a label for all equivalence Class of proofs. 
Acr~ DE COLING-92, NANTES, 23-28 AOI~'T 1992 1 0 9 8 PROC. OF COLING-92. NA~CrEs, AUG. 23-28. 1992 
I'I1 Is \] iO Sue 10 ~ S L~(s) j ~(il,,v) Lr(I J 
l~t (e--~t)-~t 
AQ.Q AP.P(sue') 
..... I s \] I 0 
) L ~-I up) j 
e--~ t 
AY.saw'(sue',Y) 
The semantic types can generally be extracted fronl 
the syntactic types. The base types s and ap map to 
the semantic types l and e, stmtding for trutb-value 
and enl, ity respectively. Categories with argmnents 
map to corresponding flmctional types. 
Provided a close mapping between syntactic and 
semantic types is assumed, the addition of semantic 
values to the axiom schemata is relatively trivial, as 
is the addition of semantic vahtes to the lexicon. For 
example, the semantic value given to the verb saw is 
AYAX.saw'(X,Y), which has type e~(e-~t). 
It is worth contrasting the approach taken here 
with two otller al)proaches to incremental interpre- 
tation. Tim first is that of Pnlman (19851. Pulman's 
approach separates syntactic and senmntie analysis, 
driving semantic combinations off the actions of a 
parser for a phrase structnre grammar. The ap- 
proach was important ill showing that hierarchical 
syntactic analysis and word by word incremental in- 
terpretation are not incompatihle. The second ap- 
proach is that of Ades and Steedman (19821 who 
incorporate conlposition rules directly into a catego- 
rim granlmar. This allows a certain amount of incre- 
mental interpretation dim to tile possibility of form- 
ing constitnents for some initial substrings, flow- 
ew:r, the incorporation of composition into the gram- 
mar itself does haw: sonic unwanted side effects when 
nfixed with a use of functions of fimctions. For exam- 
pie, if the two types, N/N and N/N are composed to 
give tile type N/N, then this can be modified by an 
adjectival modifier of type (N/N)/(N/N). Thus, the 
phrase the very old green car" can get the bracketing, 
\[the \[very \[old green\]\] car\]. Although tile Applica- 
tion schema used in DDGs does compose functions 
together, DI)Gs have identical strong generative ca- 
pacity to the LDGs they are based upon (the cov- 
erage of the grammars is identical, and tile analyses 
are ill a one-to-one correspondence). 13 
6 Applications 
So far, l)ynamic l)ependeney Grammars can be seen 
solely as a way to provide incremental parsing and ill- 
terpretation for Lexicalised l)ependency Grammars. 
As such, they are not of particular linguistic signifi- 
cance, ltowever, it is possible to use DDGs as subsets 
of ntore expressive dynamic gramnrars, where extra 
axioms and deduction rules are used to provide cov- 
erage of syntactic phenomena which are difficult to 
la'I'his is also true for dyn~anic refomlulations of extended 
versions of LDG which allow functionn of functions. 
encode lexically (e.g. coordination, topicalisation and 
extrapcsition). For example, tile following deduction 
rule (again restricted to non-empty strings), 
Co String, C,, C~C, 
Co String,, "and'~C~ 
provides all account of the syntax of non-constituent 
coordination (Milward, 1991). The sentences John 
gave Mary a book and Peter a paper" and Sue sold 
and Peter thinks Bert bought a painting are accepted 
since "Mary a book" and "Peter a paper" perform 
tile same transitions between syntactic states, ms do 
"Sue sold" and "Peter thinks Ben botlght" 
The gr,'mtmars described in this paper have been 
implemeuted in Prolog. A dynamic gramnlar based 
upon the extended version of l,I)Gs is being devel 
oped to provide incremental interpretation for the 
natural hmguage interface to a grapllics package. 
References 
Ades, A. and Steedman, M. (19721 On the Order of 
Words. Linguistics and Philosophy 4, 517-558. 
Aho, A. and Ulhnan, J. (19721 The Theory of Pars- 
ing, Traaslatton and Compiling, Volume 1:Parsing. 
Prentice-Ilall lnc, New Jersey. 
Barry, G. and Pickering, M. (19901 l)et)endcncy and 
Constituency ill Categorial Gramnlar. Ill Barry, G. 
and Morrill, G. (eds), Studies in UategoTial Gram- 
mar. Centre for Cognitive Science, University of E(I- 
inburgh. 
van Benthem, 3. (19901 General l)ynamics. ITL1 re- 
port, Anlsterdaln (to appear in Theoretical Linguis- 
tics). 
I)owty, I).R., Wall, R.F. and Peters, S. (19811 ht- 
troduction to Montague Semantics. 1).l~idel, Dor- 
drecht. 
Hays, II.G. (19641 Dependency Theory: A Formal- 
ism and Some Observations. Language 40,511-525. 
lhldson, IL (1988) Coordination and Gramnlatical 
Relations. Journal of Linguistics 24, 303-342. 
Milward, D. (19901 Coordination in an Axiomatic 
Grammar. hi Coling-90, llelsmki, vol 3,207-212. 
Milward, I). (1991) Axiomatic Grammar, Non- 
Constituent Coordination, and lnerenmntal Interpre- 
tation. PhD thesis, University of Cambridge. 
Milward, 11. (1992) Dynamic Grammars. Technical 
Report, Centre for Cognitive Science, University of 
Edinburgh. In preparation. 
Puhuan, S. (1985) A Parser Ttlat Doesn't. In 2nd 
European ACL, Geneva. 
Woods, W. (1973) An Experimental Parsing Sys- 
tem for Transition Network Grammars. In Rustin, 
R. (ed.), Natural Language Processing, Algorithmics 
Press, New York. 
Aorl!s DE COLlNG-92, NANTES, 23-28 AOUq' 1992 I 0 9 9 PROC. OF COLING-92, NAN'rI~.S, Auo. 23-28, 1992 
