TAG's as a Grammatical 
Formalism for Generation 
David D. McDonald and James D. Pustejovsky 
March, 1985 
CPTM #s 
This paper will be presented at and published in The Proceedings of the 23rd Annual 
Meeting of the Association for Computational Linguistics, July 8-12, 1985, University of 
Chicago. 
146 
I, Albstraet 
Tree Adjolni-~ Grammars, or "rAG's", (Joshi, Levy & Takahashi 1975; Joshi 1983; 
Kroch & Joshi 1985) were developed as an alternative to the standard syntactic 
formalisms that are used in theoretical analyses of language. They are att~=,~ctive 
because they may provide just the aspects of context sensitive expressive power that 
actually appear in human languages while otherwise remaining context free. 
This paper describes how we have applied the thmry of Tree Adjoinln~ Grammars 
to natural language generation. We have been attracted to TAG's because their 
central operation-the extension of an "initial" phrase ~hucture tree through the 
inclusion, at very specifically constrained locations, of one or more "auxiliary" 
trees--corresponds directly to certain central operations of our own, 
performance-oriented theory. 
We begin by briefly describing TAG's as a formalism for phrase structure in a 
competence theory, and summarize the points in the theory of TAG's that are 
germaine to our own theory. We then consider generally the position of a grammar 
within the generation process, introducing our use of TAG'S through a contrast with 
how others have used systemic grammars. This takes us to the core results of our 
paper: using examples from our research with weft-written texts from newspapers, we 
walk through our TAG inspired treatments of raising and wh-movement, and show the 
correspondence of the TAG '%djunction" operation and our "attachment" process. 
In the final section we discuss extensions to the theory, motivated by the way we 
use the operation corresponding to TAG'S" adjunction in performance. This suggests 
that the competence theory of TAG's can be profitably projected to structures at the 
morphological level as well as the present syntactic level. 
2. Tree Adjunction Grammars 
The theoretical apparatus of a TAG consists of a pr/mitively defined set of 
"elementary" phrase structure trees, a '~xking" relation that can be used to define 
dependency relations between two nodes within an elementary tree, and an "adjunction" 
operation that combines trees under specifiable constraints. The elementary trees are 
divided into two sets: init/a_l and auxiliary. Initial trees have only terminals at their 
leaves. Auxiliary trees are distinguished by having one non-terminal among their leaves; 
the category of this node must be the same as the category of the root. All 
elemental trees are "minimal" in the sense that they do not recurse on any 
non-terminal. 
A node N1 in an elementary tree may be linked (co-indexed) to a second node 
N2 in the same tree provided NI c-commands N2. Linking is used to indicate 
grammatically defined dependencies between nodes inch as mbcategorization relationships 
or filler-gap dependencies. Links are preserved (though "stretched out") when their tree 
is extended through adjunction; this is the mechanism TAG's use to represent 
unbounded dependencies. 
147 
Sentence derivations start with an initial tree, and continue via the adjunction of 
an arbitrary number of auxiliary trees. To adjoin an auxiliary tree A with root 
category X to a initial (or derived) tree T, we first select mine node of category X 
within T to be the point at which the ad~.mction is to occur. Then (1) the subtree of 
T dominated by that instance of X (call it X') is removed from T, (2) the auxiliary 
tree A b knit into T at the position where X" had been located, and (3) the subtree 
dominated by X" b knit into A to replace the second occurence of the category X at 
T's frontier. The two trees have now been merged by '%pricing" A into T, displacing 
the subtree of T at the point of the adjunction to the frontier of A. 
For example we could take the initial tree: 
IS" Wh°i does IS John like e i \] \] 
(the subscript "i" indicates that the "who" and the trace "e" are linked) and adjoin to 
it the auxiliary tree: 
\[S Bill befieves S \] 
to produce the derived tree: 
IS" Wh°i does IS Bill believe IS John likes e i \] \] \] 
Adjunction may be "constrained". The grammar writer may specify which specific 
trees may be adjoined to a given node in an elementary tree; if no specification is 
given the default is that there is no constraint and that any auxiliary tree may be 
adjoined to the node. 
2.1 Key features of the theory of TAG's 
A TAG specifies surface structure. There is no notion of derivation from deep 
structure in the theory of TAG's--the primitive trees are not transformed or otherwise 
changed once they are introduced into a text, only combined with other primitive trees. 
As Kroch and Joshi point out, this means that a TAG is incomplete as an account of 
the structure of a natural language, e.g. a TAG grammar will contain both an active 
and a passive form of the same verbal subcategorizafion pattern, without an 
theory-mediated description of the very close relationship between them. 
To our minds this is by no means a deficit. The procedural machinery that 
generative grammars have traditionally carried with them to characterize relations like 
that of active to passive has only gotten in the way of employing those 
characterizations in processing models of generation. This is because a generation 
model, like any theory of performance, has a procedural structure of its own and 
cannot coexist with an incompatible one, at least not while still operating efficiently or 
while retaining a simple mapping from its actual machine to the virtual machine that 
its authors put forward as their account of psycholinguistic data. 
Our own generator uses surface siJucture as its only expficitly represented linguistic 
level. Thus grammatical formalisms that dwell on the rules governing surface form are 
more useful to us than those that hide those rules in a deep to surface 
transformational process. 
148 
¢ 
A TAG Involvw the tmmlpuladon of very small dementary structmrw. This is 
because of the stip,,Indon that elementary trees may not induck recurtive nodes. It 
impliea that the sentences one tee, in everyday usage, e.g. newpaper texts, are the 
result of many tucemive adjunctions. This melds nicely with a move that we have 
mack in recent yeatt to view the conceptual representation from which generation 
proceeds as comiging of a heap of very small, reduadaatly related information units 
that have been deliberatebj selected by a text planning process from the total state of 
the kaowledge base at the time of utterance; each satch unit will correspoad in the 
final text to a head lexical item plus selected thematic arguments--a linguistic entity 
that is easily projected onto the elementary trees of a TAG. 
TAG theory ladmles only one operation, adJnnctlon, and otherwise makes no 
changes to the elementary trees that go into • text, This comports well with the 
indefibility stipulation in our model of generation, dnce selected text fragments can be 
used directly as specified by the grammar without the need for any later 
transformation. The composition options delimited by the constraints on adjunction 
given with a TAG define a space of alternative text forms which can correspond 
directly in generation to alternative conceptual relations among information units, 
alternatives in rhetorical intent, and alternatives in prose style. 
3. Adapting TAG's to Generation 
The mapping from TAG's as a formalism for competence theories of language to 
our formalism for generation is strikingly direct. Their adjuncti0n operation corresponds 
to our "attachment process"; their constraaints on adjunction correspond to our 
"attachment points"; their surface structure trees correspond to our surface structure 
trees. 1 We further hypothesize that two quite strong correspondence claims can be 
made, though considerably more experimentation and theorizing will have to be done 
with both formalisms before these claims can be confirmed. 
1. The primitive information units in realization specifications can be realized 
exclusively as one or another elementary tree as defined by a suitable TAG, 
i.e. linguistic criteria can be used in determining the proper modularity of the 
conceptual structure. 2 
2. Conversely, for any textual relationship which our generator would derive by 
the attachment of multiple information units into a tingle package, there is a 
corresponding rule of adjunction. Since we use attachment in the realization 
of nominal compouads like "o// tanker", this has the force of extending the 
domain of TAG analyses into morphology. (See section 7). 
1 Our model of generation does not employ the ample trees of labeled nodes that appear in most 
theoretical linguistic amtlytet Our turfa~ ttructurc incorporates the igmantic propertim of trees, but 
it also incJ,vt-,, rcifx=ttions of consdtmmt positions like "subject" or "acntem:e" and is bcttg~r 
characterized overall as an "executable aequence of labeled positions". We discuss this further in 
::section 5.1. 
21f this iqq:x)tlm is mcemfui, it has very comequenfial ;mnlicadonl for thz "~.¢" of the 
information units that tbe text planner constructing tim realization specification can me, e.g. they would 
not be r~y.,~t at texts that include recmtiv, nodes. We will discuss this and other implications in • 
later ~. 149 
4. l\[ e Place of Gramnmr in a Theory of Generation 
To understand why we are looking at TAG's rather than some other formalism, 
one must first understand the role of grammar within our processing model. The 
following is a brief summary of the model; a more complete description can be found 
in McDonald & Pustejovsky \[1985b\]. 
We have always had two complementary goals in our research: on the one hand 
our generation program has had to be of practica~l utility to the knowedge based expert 
systems that use it as part of a natural language interface. This means that 
architecturally our generator has always been designed to produce text from conceptual 
specifications, ~plans", developed by another program and consequently has had to be 
sensitive to the limitations and varying approaches of the present state of the art in 
conceptual representation. 
At the same time, we want the architecture of the virtual machine that we 
abstract out of our program to be effective as a source of psycholinguistic hypotheses 
about the actual generation process that humans use; it should, for example, provide 
the basis for predictive accounts of human speech error behavior and apparent planning 
limitations. To achieve this, we have restricted ourselves to a highly constrained set of 
representations and operations, and have adopted strong and suggestive stipulations on 
our design such as high locality, information encapsulation, online quasi-realtime runtime 
performance, and indelibility) This restricts us as programmers, but disciplines us as 
theorists. 
We see the process of generation as involving three temporally intermingled 
activities: (1) determining what goals the utterance is to achieve, (2) planning what 
information content and rhetorical force will best meet those goals given the context, 
and (3) realiz/ng the specified information and rhetorical intent as a grammatical text. 
Our linguistic component (henceforth LC), the Zetalisp program MUMBLE, handles the 
third of these activities, taking a "realiTztion specification "4 as input, and producing a 
stream of morphologically specialiTed words 5 as output. 
As described in \[McDonald 1984\], our LC is a "description-directed" process: it uses 
the structure of the realization specification it is given, plus the syntactic surface 
structure of the text in pro~ss (which it extends incrementally as the specification is 
realized) to directly control its actions, interpreting them as though they were sequential 
computer programs. This technique imposes strong demands on the descriptive 
formalima used for representing surface structure. For example, nodes and category 
labeis now designate actions the generator is to take (e.g. imposing scoping relations 
or constraining embedded decisions) and dictate the inclusion of function words and 
morphological specializations. 
3 "~ty" in a computation requires that no action of a process (making decisions, constructing 
represeatations, chan8~ state, etc.) can be tramparently undone once it has been performed. Many 
nonbacktracking, nonfntra~ program designs have this property; it is our term for what Marcus \[1980\] 
referred to as the property of being "strictly determlnhtic'. 
4A realization specification can informally be taken to correspond to what many researchers, 
partioti~iy p~hok~t.% lhink of Its the "me~tge love.J" g~Uttion of a text. 
5 Which is to say tb~t it presently produces written rath~ .h n ~ok~ U~UI. We e~rp¢~ tO work 
with speech output shortly, howev~, sad the need to support the ~tational basis of an 
intonational contour is begi~n£ to influence our dcsigns for constituency patterns in surface structure. 
150 
4J Unb d s Gmm.n 
Of the egablished linguistic formalisms, systemic grammar \[Halliday 1976\] has 
always been the most important to AI researchers on generation. Two of the most 
important generation systems that have been developed, PROTEUS \[Davey 1974\] and 
NIGPJ- \[Mann & Matthieuen 1983\], use systemic grammar, and others, including our 
own, have been strongly influenced by it. The reasons for this enthusiamn are central 
to the special concerns of generation. Systemic grammars employ a functional 
vocabulary: they emphasize the uses to which language can be put--how languages 
achieve their speakers" goals-rather than its formal structure. Since the generation 
process begins with goals, unlike the comprehem/on process which begins with structure, 
this orientation makes systemic grammars more immediately useful than, for example, 
transformational generative grammars or even procedurally oriented AI formalisms for 
language such as A TN's. 
The generation researcher's primary question is why use one construction rather 
than another--active instead of passive, "the" instead of "a". The principle device of a 
systemic grammar, the "choice sy~em", supports this question by highlighting how the 
constructions of the language are grouped into sets of alternatives. Choice systems 
provide an anchoring point for the rules of a theory of language use since it is natural 
to a~ociate the various semantic, discourse, or rhetorical criteria that bear on the 
selection of a given construction or feature with the choice system to which the 
construction belongs, thus providing the basis of a decision-procedure for selecting from 
its listed alternatives; the NIGEL system does precisely this in its "chooser" procedures. 
In our formalism we make use of the same information as a systemic grammar 
captures, however we have choosen to bundle it quite differently. The underlying reason 
for this is that our concern for psycholinguistic modeling and efficient processing takes 
precedence in our design decisions about how the facts of language and language use 
should be represented in a generator. It is thus instructive to look at the different 
kinds of linguistic information that a network of choice systems carry. In our system 
we distribute these to separate computational dev/ces. 
o Dependencies among structural features: A generator must respect the 
constraints that dependencies impose and appreciate the impact they have on its 
realization options: for example that some subordinate clauses can not express 
tense or modality while main clauses are required to; or that a pronominal 
direct object forces particle movement while a lexical object leaves it optional. 
o Usage criteria. The decision procedures amoc/ated with each choice system are 
not a part of the grammar per se, although they are naturally associated with it 
and organized by it. Also most systemic grammars include very abstract 
featur~ such as "generic reference" or "completed action", which cross-correlate 
the language's surface features, and thus are more controllers of why a construct 
is used rather than conshucts themselves. 
o Coordinated structural alternatives. A sentence may be either active or passive, 
either a question or a statement. By groupin 8 these alternatives into systems 
and using these systems exdusively when constructing a text, one is guaranteed 
not to combine inconsistent structural features. 
"7 
o Efficient ordering of choices. The network that connect, choice t~stems provides 
a natural path between decisiom, which if followed strictly guarentees that a 
choice will not be made unlm it is required, and that it will not be made 
before any of the choic_~___ that it it itself dependent upon, insuring that it can 
151 
be made indelibly. ~ 
o Typology of sm'face structure. Almost by accident (dnce its specification is 
dhgtributed throughout all of the systems impficitly), the grammar determines the 
pattern of dominance and comtituency relationshiln of the text. While not a 
principle of the theozy, the trees of clanses, NP% etc. in systemic grammars tend 
to be shallow and broad. 
We believe, but have not yet established, that equivalence transformations can be 
defined that would take a systemic grammar as a q3ecification to construct the 
alternative devices that we use in our generator (or auement devices that derive from 
other sources, e.g. a TAG) by decomposing the information in the systemic grammar 
along the lines just listed and redistributing it. 
5. Example Analyses 
One of the task domains we are currently developing involves newspaper reports of 
current events. We are "reverse engineering" leading paragraphs from actual newspaper 
articles to produce narrow but complex conceptual representation, and then desiEning 
realization specifications--plans--that will lead our LC to reconstruct the original text or 
motivated variations on it. We have adopted this domain because the news reporting 
task, with its requirement of communicating what is new and si!,nificant in an event as 
well as the event itself, appears to impose exceptionally rich constraints on the selection 
of what conceptual information to report and on what syntactic constructions to use in 
reporting it (see discussion in Ciipplnger & McDonald \[1983 D. We expect to fred out 
how much complexity a realization specification requires in order to motivate such 
carefully composed texts; this will later guide us in designing a text planner with 
sufficient capabilities to construct such specifications on its own. 
Our examples are drawn from the text fragment below (Associated Press, 12/23/84); 
the realization specification we use to reproduce the text follows. 
"LONDON - Two oil tankers, the Norwegian-owned Thorshavet end a 
Uben'en-registered v~__el, were reported to have been hit by missiles Friday in the Guff. 
The Thorshavet was ablaze end under tow to Bahrein, officiels in Oslo said. 
Uoyds reported that two crewmen were Injured on the Ubenan ship." 
(the-day" Hvents-~-the-GU~-tanker-war 
~1 as-ttHiource 
(mn~-ever4 #~me-ever~-~lpe_vnryk,o-om{em 
#<hlbby-rnbalm Thomhavel> 
#<~t-by-mbsaes Ubedan> > 
~nmual ~<nurrter-of-sP~ps-I~a 2> Jde y-the- ps ) 
(partkxlars #<damage-r~xxt Thorahavet O~o-omc~s> 
#<da.~e-repo,t Uber~n Uoyds> )) * 
152 
This realization speOJicatiou represents the structured object which gives the 
toplevel plan for this utterance. Symbols preoeded by colons indicate particular features 
of the utterance. The two expressions in parentheses are the content items of the 
specification and are restricted to appear in the utterance in that order. The first 
symbol in each expr~on is a label indicating the function of that item within the 
plan; embedded items appearing in angle brackets are information units from the 
current-events knowledge base. 
Obviously this plan must be considerably refined before it could serve as a 
proximal source for the text; that is why we point out that it is a "toplever' plan. It 
is a specification for the general otltline of the utterance which must be fleshed out by 
recursive planning once its reafization has begun and the LC can supply a linguistic 
context to further constrain the choices for the units and the rhetorical features. 
For present purposes, the key fact to appreciate about this realization specification 
is how different it is in form from the surface structure. One cannot produce the 
cited text simply by traversing and "reading out" the elements of the specification as 
though one were doing direct production. Structural rearrangements are required, and 
these must be done under the oontrol of constraints which can only be stated in 
linguistic vocabulary with terms like "subject" or "raising". 
The first unit in the spcc/fication, ~<same-event-t~e_>, is a relation over two 
other units. It indicates that a commonality between the two has been noticed and 
deemed significant in the underlying representation of the event. The present LC 
always realizes such relations by mergil~ the realizations of the two units. If nothing 
else occurred, this would give us the text "Two oil tankers were hit by missiles". 
As it happens, however, a pending rhetorical constraint from the realization 
specification, ~svents-r~ulre-¢erUfk~on-u4o-source will force the addition of yet another 
information unit, 6 the reporting event by the news serv/ce that announced the aledged 
event (e.g. a press release from Iraq, Reuters, etc.). In this case the "coatent" of the 
reporting event is the two damage-repor~ which have already been planned for inclusion 
in the utterance as part of the '~trticulars" part of the specification. Let us look 
closely at how that reportiing event unit is folded into surface structure. 
When not itself the focus of attention, a reporting event is typically realized as 
'~u3-and-so said X", that is, the content of the report is more important than the report 
itself; whatever significance the report or its source has as news will be indicated subtly 
through "which of the alternative realizations below is selected for it. 7 
6 We will not discuss the mechanism by which features in the specification inlluetg~ realization. 
Realization qgcifi~fions of the complexity of this emsmpiz are still very new in our research and we 
arc unsure whether the process is better organized at the mnceptual level directing • composition 
process within the planning componeat (during one of the recursive invocations) or within the LC 
.. mediating a selection between anticipated alternatives. At this point our design experiments are 
• inconclusivc. 
7 These sentenees are s.rfificial; meal ones would be considerably longer. Interestingly, o:rtain 
other syntactically permissable ~,~,~oes such ss "/t WaS ~oorted that" do not occur in say of the 
texts we have examined. Perhaps the "lead ~ position is too tmporUmt to waste on a proneun. 
153 
nedred cbmu er ae Rmdt  text 
de.empha~ report Two tankers were hit, Gulf shippin& sources said. 
source b 0yea ebewhere Two tankers were reported hit. 
emiMmMxe report Iraq reported it hit two tankers. 
Figure 2 Possibilities for expressing report(source, info) in newpaper prose 
In our LC, these alternative "choices" are grouped together into a "realiTation 
class" as shown in F'tgure 3. Our realiTafion classes have their historic orisin5 in the 
choice t3nltem$ of systemic grammar, though they are very different in almost every 
concrete detail. The most important difference of interest theoretically is that while 
wstemic choice systems select among single alternative features (e.g. p~ve, gerundive), 
realization classes select among entire surface structure fxa~,ments at a time (which 
might be seen as prespecified realizations of bundles of features). That is, our 
approach to generation calls for us to organize our decision procedures so as to select 
the values for a number of linguhtic features simultaneously in one choice where a 
systemic grammar would make the selection incrementally. 8 
8 The standard technique of using choicc systems to control ~ activc ~.Icction of utterance 
features b employed by ~ most well-known applications of systa:nic grammars to ge.J,~ation (i.e. the 
work of Davey \[1974\] and Mann and Matthiessen \[1983D. Howover very reo~t work with systemic 
grammars at Edinburgh by Patten \[1985\] depa~ from this technique. Patten uses a semantic-level 
plannin~ coml~lte~t to directly ~ grOUpl of features at the rightward, "output", rode of a systemic 
network, and then works hackwarde through the network to determine what other, not semaficcaUy 
features must be added to the text for it to be grammatical; control is thus outside the 
grammar proper, with gramnunr rule, rckqptted to constraint specification only. We are intrigued by 
this technique and look forward to its further dcvelopmcnt. 
154 
beleve-vedm 
: parameters (aOent propomon ve, b) 
: cholcem 
; e.g. =L/oyds reports lraq hit two tankers." 
; encompasses variations with and without that, and 
; also tenseless complements like =John believes him 
: to be a fool." 
( (raise-VERB-Into-PROP (passlvlze verb| prop) dame focus((aOe.t prop)) menUoned4sewtmm(aoent)) 
; "Two tankers were reported to have been hit" 
( (It-VERB-PROP ve, b prop) dame ~erat~e(agent) ) 
; e.g. =it is reported that 2 tankers were hit." 
( Oeft,.dlslocated~ agent ved) prop) ciauu de-empemize(se~ 
; "Two tankers were hit, Gulf sources saM." )) 
Figure 3 Realization class assigned to report(l~J~t(_) 
Returning to our example, we are now faced now with the need to incorporate a 
unit denoting the report of the Iraqi attacks into the utterance to act as a certification 
of the #<~-by-rmsaes> events. This will be done using the realization class 
be/ieve-vedm; the class is applicable to any information unit of the form report(source, 
info) (and others). It determines the realization of such units l?oth when they appear in 
issolation and, as in the present case, when they are to augment an utterance 
corresponding to one of their arguments. 
From this realization class the choice raise-VERB-In, PROP will be selected since (1) 
the fact that two ships were hit is most ~gnificant, meaning that the focus will be on 
the information and not the source (n.b. when the class executes the source t'aq will be 
bound to the a0ent parameter and the information about the missile hits to the 
proposaM, n parameter); (2) there is no rhetorical motivation for us to occupy space in 
the tint sentence with the sources of the report since they have already been planned 
to follow. These conditions are sensed by attached pxocedures associated with the 
characteristics that annotate the choice (i.e. focta and menti0md-elsewhem). 
Since the PROP is ah'cady in place in the mrface structure tree, the LC will be 
interpreting ralse-VERB.Inlo-PROP as a q)eciflcation of how it may fold the auxiliary tree 
for reported into the tree for Two oil tankers were hit by missiles Friday in the Gulf. 
This corresponds to the TAG analym in F'tgure 4 \[Kroch & Joshi 1985\]. 
155 
Initial Tree: 
c ,J 
NP INFL 
two tankers / 
INFL VP 
be hit by mi~iles 
Auxiliary Tree: 
INFL 
INFL VP 
be reported INFL 
Figure 4 In/tiaI and amdUlary trees for RalMng4e.eubject 
The initial tree for Two oil tankers were hit by missiles, I1, may be extended at its 
INFL" node as indicated by the constraint given in parenthesis by that node. Figure 5 
shows the tree after the auxiliary tree A 2, named by that constraint, has been adjoincd. 
Notice that the original INFL" of Figure 4 is now in the complement position of 
report, giving us the sentence Two oil tankers were reported hit by missiles. 
NP j----._.. 
two mi~eilee 
INFL 
INFL VP 
be reported IIq'FL 
INFL VP 
be hit by missiles 
Figure $ After embedding report 
5.I Path Notation 
As readers of any of our earlier papers arc aware, we do not employ a 
conventional tree notation in our LC. A generation model places its own kinds of 
demands on the representation of surface structure, and these lead to principled 
departures from the conventions adopted by theoretical linguists. Figure 6 shows the 
surface structure as our LC would actuary represent it just before the moment when 
the adjunction is made. 
156 
\[SENTENCE\] .... > ... 
\[SUBJECTI 
(plural) 
\[qaent\] ~ \[heedl 
t,#o N 
\[l,r,moal \[~ea\] 
oil tanker 
J IPBE DICATE\] 
O Attach- 
o¢ltit-l~y -missiles..., Baieit~- Precticete 
Figure 6 Surfam structure in pathnotation 
We call this representation path notat/on because it defines the path that our LC 
follows. Formally the structure is not a tree but a unidirectional linked list whose 
formation rules obey the axioms of a tree (e.g. any path "down" through a given node 
must eventually pass back "up" through that same node). The path consists of a 
stream of entities representing phrasal nodes, constituent positions (indicated by square 
brackets), instances of information units (in boldface), instances of words, and activated 
attachment points (the labeled circle under the predicate; see next section). The 
various symbols in the figure (e.g. sentence, predicate, etc.) have attached procedures 
that are activated as the point of speech moves along the path, a process we call 
'~phrase ~iJucture execution". Phrase ttlucture execution is the mean.~ by which 
grammatical constraints are imposed on embedded decisions and function words and 
grammatical morphemes are produced. (For discussion see McDonald \[1984\].) 
Once one has begun to think of surface structure as a travenal path, it is a short 
step to ima~inln~ being able to cut the path and '~plice in" additional position 
sequence. 9 This ~licing operation inherits a natural set of constraints on the ldnds of 
distortions that it can perform, since, by the indelibility stipulation, existing position 
sequences can not be destroyed or rethreaded. It is our impression that these 
constraints will turn out to be formally the same as those ~ of a TAG, but we have not 
yet carried out the detailed analyses to confirm this. 
9The fem-ueity of cutting the mzrface structure and insetting new eequences that change the 
• linguistic context of pmitiom akeady in place has been in our thmry of generation since 1978, when 
we used it to implement rat~Ing veto whooe rhetorical fogee was the mine as "hedging" advedbs like 
poax//dy. Our pre~at, much more e~mtve use of thia device as the core of a distim:t attachment 
procem dates from the summer of 1984. 
157 
$.2 Attaehm_m,t Peints 
The TAG formalism allows a grammar writer to define "constraints" by annotating 
the nodes of elementary trees with lists indicating what auxiliary trees may be adjoined 
to them (including "any" or "none"). l° In a similar manner the "choices" in our 
realization cl~which by our hypothesis can be taken to always correspond to TAG 
elementary trees--include specifications of the attachment po/nts at which new 
information units can be incorporated into the surface structure path they defme. 
Rather than being constraints on an otherwise freely applying operation, as in a TAG, 
attachment points are actual ob~cts interposed in the path notation of the surface 
structure. A list of the attachment points active at any moment is maintained by the 
attachment process and consulted whenever an information unit needs to be added. 
Most units could be attached at any of several points, with the decision being made on 
the basis of what would be most consistent with the desired prose style (cf. McDonald 
and Pustejovsky \[1985aD. When one of the points is selected it is instantiated, usually 
splicing in new surface structure in the process, and the new unit added at a 
designated position within the new structure. Figure 7 shows our present definition of 
the attachment point that ultimately leads to the addition of "was reported". 
(define-attachment-point attach-raJs~ng.~'edcate reference-poinm 
( (~"~l~.~elot °pr~.ate phrase) zttach-under ) 
(vp-lrdinltlve-cong31emeot) ; specification of new phrase 
verb ; where the unit being attached goes 
~inttJve-cor~) ; where the existing contents go 
e~ezt~-oew-peod~a~-polnts 
cholcm-tl~-Introduc~lt 
x~oeee-~te~ ~et~t "prececate)) 
Figure 7 The attltchmeat-lpoint wed by was reported 
10 Constraints of this sort are an inovation introduced in Kroch & Joshi \[1985\]. 
Previous versions of TAG theory allowed "context sensitive" constraint specifications 
that in fact were never exploited. The present constraints are more attractive formally 
since they must be stated locally to a single tree. 
158 
This attachment point goes with any choice (elementary tree) that includes a 
constituent position labeled ia~dUkate. It is placed in the position path immediately 
after (or "under") that position (see Figure 6), where it is available to any new unit 
that passes the indicated requirements. 
When this a.~chment is selected, it builds a new VP node that has the old VP as 
one of its comtituents, then spUces this new node into the path in its place as shown 
in F~u,~ 7. 
The unit being attached, e.g. the report of the attack on the two oil tankers, is 
made the verb of the new VP. Later, once the phrase structure execution process has 
walked into the new VP and reached that verb position, the unit's realization class 
(belief-verbs) will be consulted and a choice selected that is consistent with the 
grammatical constraints of being a verb (i.e. a conventional variant on the 
raise-VERB4nto-PROP choice), 8ivi~ us "wos reported". 
\[SUBJECT\] 
NP 
t~,o oil tankers 
• > \[SENTENCE\] ~ ..: 
S 
, \[PREDICATE\] 
\[verb\] ---> \[|nfiniUve- 
rel~or t complement\] 
-<hit-by-mi--ile----> 
Figure 8 'nze path after" attachment 
From this discussion one can see that our treatment of attachment uses two 
gructures, an attachment point and a choice, where a TAG would only use one 
structure, an auxiliary tree. ~ is a consequence of the fact that we are working 
with a performance model of generation that must show explicitly how conceptual 
information units are rendered into texts as part of a psycholinguisticaUy plausible 
process, while a TAG b a formalism for competence theories that only hoe4 to specify 
the syntactic itmcture of the grammatical strings of a language. This is a significant 
difference, but not one that should stand in our way in comparing what the two 
theories have to offer each other. Comequently in the rest of this paper we will omit 
the d~aill of the path notation and attachment point def'mitions to facilitate the 
comparison of theoretical issues. 
159 
6. Generating qua/ions m/ng a TAG version of wh-movement 
Earlier we illustrated the TAG concept of "linking" by showing how one would 
start with an initial tree consisting of the /nnermost clause of a question plus the 
fronted wh-phrase and then build outward by successively adjoining the desired auxiliary 
phrases to the S node that intervenes between the wh-phrase and the clause. 
Wh-questions are thus built from the bottom up, as in fact is any sentence involving 
verbs taking sentential complements. 
analylis has the desirable property of allowing one to state the dependendes 
between the Wh-phrase and the gap as a local relation on a single elementary tree, 
eliminating the need to Include any machinery for movement in the theory. All 
unbounded dependendes now derive from adjunctions (which, as far as the grammar is 
concerned, can be made without limit), rather than to the expficit migration of a 
constituent acrms clauses. 
We also find this locality property to be desirable, and use an analogous procedure 
in our production of questiona and other kinds of Whquestions and unbounded 
dependency constructions. 
This "bottom-up" design has consequences for how the realization specifications for 
these constructions must be organized. In particular, the logician's usual representation 
of sentential complement verbs as higher operators is not tenable in that role. For 
example we cannot have the source of, say, How many ships did Reuters report that 
Iraq had said it attacked? be the expreuion: 
Lambda(quantlty-of-shlps) . report(Reuters, sty(Lraq, attack(Iraq, quantity-of-ships))) 
Such an expression defines a natural sequence of exposure when used as realization 
specification, namely that one realize the Lmnlght operator first, the report operator 
second, the sty third, and so on. A local TAG analysis of Wh-movement requires us 
to have the Lambda and the expression containing its matrix trace, attach, be present 
in a single "layer" of the specification, otherwise we would be forced to violate one of 
the strong principles of our theory of generation, namely that the characteristics in a 
realization class may "see" only the immediate arguments of the unit being realized; 
they may not' look "inside" those arguments to subsequent levels of conceptual 
structure. 
This principle has served t~s well, and we are disinclined to give it up without a 
very compelling reason. We elected instead to give up the internal representation of 
sentential complement verb texts as tingle expressions. This move was easy for us to 
make since such expreuions are awkward to manipulate in the "East Coast" style frame 
knowledge bases that we use in our own reasonin S programs, and we have preferred a 
representational style with redundant, smaller sized conceptual units for quite some 
time. 
The representation~ we use instead amounts to breaking up the logical expression 
into individual units and allowing them to include references to each other. 
U I = lambda(quantity-of-shipa) . attack(Iraq,quantity-of-ships) 
U 2 = sty(Iraq, U1) 
u 3 : repo.(Reuten, U2) 
160 
Given such a network as the realization specification, the LC must have some 
principle by which to judge where to start: which unit should form the basis of the 
surface mucture to which the others are then attached? A natural principle to adopt 
is to begin with the '°rash" unit, i~. the one that does not mention any other units in 
its definition. We are comiderin s adopting the policy that such units should be 
allowed only realizations as initial trees while units whose definition involves "pointing 
to" (naming) other units shouldbe allowed only rea!iza" tions as auxiliary trees. We 
have not, however, worked through all of the ramifications such a policy might have 
on other parts of our generation model; without yet knowing whether it would improve 
or degrade the other parts of our theory, we are reluctant to assert it as one of our 
hypotheses relating our generation model to TAG's. 
Given that three part source, the realization of the question is fairly 
straightforward (See Figure 9). The Lambda expression is assigned a realization class 
for clausal Wh constructions, whereupon the ©xUscted argument quantlty.of.shlps is 
placed in COMP, and the body of the expression is placed in the HEAD position. At 
the same time, the two instances of qumd/ty-of-ehIp8 are specially marked. The one in 
COMP is ass/gned to the realization class for Wh phrases appropriate to quantity (e.g. 
it will have the choice how many X and possibly related choices such as <quantity> of 
which and other variants appropriate to relative clauses or other positions where Wh 
constructions can be used). Simultaneously the instance of quanflty-ef-eidps in the 
argument position of the head frame attack is assigned to the realization class for 
Wh-trace. These two specializations are the equivalent, in our model, of the TAG 
linking relation. 
~ ~" Reuters reports S 
Iraq say S comp / 
WH(ships) 
° 
Fitmre 9 Qeesaea fermae\[oa with ~ ¢emplement verbs 
The two pending units, U 2 and U3, are then attached to this matrix, submerging 
first the attach unit and then U 2 into complement positions. 
161 
7. Extensions to'the 'l\]~eory of TAG 
Context-free grammars are able to express the word formation processes that seem 
to exist for natural language, (cf. Williams \[1981\], Selkirk \[1982\]). A TAG analysis of 
inch a grammar seems like a natural application to the current version of the theory 
(d. Puste~k-y (in preparation)). To illustrate our point, consider compounding rules 
in English. We can say that for a context-free grammar for word formation, Gw, there 
is a TAG, T w, that is equivalent to G w (d. Figur~ 10 and 11). Consider a fragment 
of G w below. It 
N->N IA I V I P H 
A->NIAIP A 
V->PV 
Figure 10 CFG Fragment for Word Formation 
The corresponding G w fragment would be: 
N 
comp N 
A 
comp A P V 
AUXI LIARY TREES 
N N N I I I 
oil tanker port 
INITIAL TREES 
Figure 11 TAG Fragment for Word Formation 
Now conmder the compound , =oil tanker terminal, taken from the newspaper reporting 
domain, and its derivation in TAG theory, shown in Figure 12. 
11 Whether the word formation component should in fact have the power of a TAG 
or CFG is an open question. Langendoen \[1981\] discusses the possibility that a f'mite 
state grammar might be sufficient for the generative capacity of natural language word 
formation components. 
162 
ogre( ~N /~anker ~k k ~e~inal 
oil tanker 
Figure 12 TAG Derivation of oil tanker terminal 
Let us compare this derivation to the process used by the LC. The underlying 
information units from which this compound is derived in our system are shown below. 
The planner has decided that the units below need to be communicated in order to 
adequately express the concept. The top-level unit in this bundle is #<terminal>. 
U 1 = #<M~rn~u~> 
O 2 = #<aoekn-at U I th> 
u s = #<tank~> 
0 4 = #<~,,~,, L h us> 
U s = #<o,> 
The first unit to be positioned in the mrface structure is U 1, and appears as the head 
of an NP. There is an attachment point on this position, however, which allows for the 
possibility of expressing U 2 prenominally. One of the choices associated with this unit is 
a compound structuro--expre~ed in terms of an auxiliary tree. A snapshot at this point 
in the derivation shows the following structure. 
\[C~,.~ U2 \] Ol \] 
The next unit opened up in this structure is U 3, which also allows for attachment 
prenominally. Thus an auxiliary tree corresponding to U 4 is introduced, giving us the 
structure below: 
~ ~.~ u4 \] u~ u 11 
The selectional constraints imposed by the structural positioning of information unit 
U 4 allows only a compounding choice. Had there been no word-level compound 
realization option, we would have worked our way into a corner without expressing the 
relation between #<oil> and #<tanker>. Because of this it may be better to view 
units such as U 4 as being associated directly with a lexical compounded form, i.e. oil 
tanker. This partial solution, however, would not speak to the problem of active word 
formation in the language. Furthermore, it would be interesting to compare the 
strategic decisions made by a generation system with those planning mistakes made by 
humans when speaking. This is an aspect of generation that merits much further 
research. 
163 
8. Acknowledgements 
This research has been mpterminaled in part by contract N0014-85-K4)017 from the 
Defense Advanced Research Projects Agency. We would like to thank Marie Vaughan 
for help in the preparation of this text. 
9. References 
Clippinger, & McDonald (1983) "Why Good Writing is Easier to Understand", Proc. 
IJCAI-83, pp. 730-732. 
Davey (1974) Dbcourse Production, Ph.D. Dissertation, Edinburgh University; publi~ed 
in 1979 by Edinburgh University Press. 
Halliday (1976) System and Function in Language, Oxford University Press. 
Joshi (1983) "How Much Context-Sensitivity is Required to Provide Reasonable 
Structural Descriptions: Tree Adjoining Grammars", preprint to appear in Dowty, 
Karttunen, & Zwicky (ecls.) Natural Language Processing: Psycholin=~2ulstlc , 
Comimtatlonal, and Theoretical Perspectives, Cambridge University Press. 
Kroch, T. and A. Joshi (1985) "The Linguistic Relevance of Tree Adjoining Grammar", 
University of Penosylvania, Dept. of Computer and Information Science. 
Langendoen, D.T. (1981) 'q'he Generative Capacity of Word-Formation Components", 
Linguistic Inquiry, Volume 12.2 
Mann & Matthiessen (1983)Ni&el: A Systemic Grammar for Text Generation, in Freedle 
(ed.) Systemic Perspectives on Discourse, Ablex. 
Marcus (1980) A Theory of Syntactic Recognition for Natural Language, MIT Press. 
McDonald (1984) "Description Directed Control: Its Implications for Natu~ral Language 
Generation", in Cercone (ed.) Cemlmtatlonal IAngulstlcs, Pergamon Press. 
McDonald & Pustejovsky (1985a) "SAMSON: a computational theory of prose style in 
generation", Proceedings of the 1985 meeting of the European Association for 
Computational Linguistics. 
(1985b) "Description-Directed Natural Language Generation", Proceedings 
of IJCAI-85, WXaufmann Inc., Los Altos CA. 
Patten T. (1985) "A Problem Solving Approach to Generating Text from Systemic 
Grammars", Proceedings of the 1985 meeting of the European Association for 
Computational Linguistics. 
Pustejovtky, J. (In Preparation) "Word Formation in Tree Adjoining Grammars" 
Selkirk (1982) The Syntax of Words, MIT Press. 
Williams (1981) "Argument Structure and Morphology" The Linguistic Review, 1, 81-114. 
164 
