<?xml version="1.0" standalone="yes"?>
<Paper uid="H90-1010">
  <Title>Two Recent Developments in Tree Adjoining Grammars: Semantics and Efficient Processing</Title>
  <Section position="1" start_page="0" end_page="52" type="abstr">
    <SectionTitle>
ABSTRACT
</SectionTitle>
    <Paragraph position="0"> During the past year there have been two very significant developments in the area of Tree Adjoining Grammars (TAGs).</Paragraph>
    <Paragraph position="1"> The first development is a variant of TAGs, called synchronous TAGs, which allows TAG to be used beyond the confines of syntax by characterizing correspondences between languages. The formalism's intended usage is to relate expressions of natural languages to their associated semantics represented by a logical form language in TAG, or to their translates in another natural language. The formalism is incremental and inherently nondirectional. We will show by detailed examples the working of synchronous TAGs and some of its applications, for example in generation and in machine translation.</Paragraph>
    <Paragraph position="2"> The second development is the design of LR-style parsers for TAGs. LR parsing strategies evolved out of the original work of Knuth. Even though they are not powerful enough for NLP, they have found use in natural language processing 0VLP) by solving by pseudo-parallelism conflicts between multiple choices. This gives rise to a class of powerful yet efficient parsers for natural language. In order to extend the LR techniques to TAGs it is necessary to find bottom-up automaton that is exactly equivalent to TAGs.</Paragraph>
    <Paragraph position="3"> This is precisely what has been achieved by the discovery of the Bottom-up Embedded Push Down Automaton (BEPDA).</Paragraph>
    <Paragraph position="4"> Using BEPDA, deterministic left to fight parsers for the Tree Adjoining Languages have been developed.</Paragraph>
    <Paragraph position="5"> Using TAGs beyond their Role in Syntax null The unique properties of tree-adjoining grammars (TAG) present a challenge for the application of TAGs beyond the limited confines of syntax, for instance, to the task of semantic interpretation or automatic translation of natural language. A variant of TAGs, called synchronous TAGs, has been developed (Shieber and Schabes \[1990(a)\]). It is used *This research is partiaUy supported by Darpa grant N0014-85-K0018, ARO grant DAAL03-89-C-0031PRI and NSF grant-IR184-10413 A02. Parts of this work are results of coUaborafion with Stuart Shieber (Harvard University), K. Vijay-Shanker (University of Delaware) and Anne Abei116 (University of Paris-7). The specifics of the collaboration are stated in the body of the paper. We are also grateful to Bernard Lang and David Weir for their valuable suggestions on LR-style parsing of TAGs which played an instrumental role in the definition of BEPDA, for example restriction on the moves allowed.</Paragraph>
    <Paragraph position="6"> to relate expressions of natural languages to their associated semantics represented in a logical form language or to their translates in another natural language (the work on Synchronous TAG and its applications to language interpretation and generation has been done in collaboration with Stuart Shieber).</Paragraph>
    <Paragraph position="7"> Language interpretation and generation with TAGs The key idea for semantic interpretation is that the logical form language itself can be described by a TAG. The two TAGs (one for the natural language and one for the logical form language) work synchronously, in the sense that the certain correspondences (links) are stated initially between the elementary trees of the two TAGs and then composition operations (such as substitution and adjoining) are carried out synchronously on the linked nodes of the two TAGs.</Paragraph>
    <Paragraph position="8"> The fact that both the natural language and the logical form language can be described by TAGs is a direct consequence of the extended domain of locality of TAGs as compared to LFG or GPSG.</Paragraph>
    <Paragraph position="9"> A sample synchronous TAG is given in Figure 1. Each element of the synchronous TAG is a pair consisting of two elementary trees, one from the source language (English) and one from the target (logical form \[LF\]). Nodes, one from each tree, may be linked; such links are depicted graphically as thick lines. If we project the pairs onto their first or second components (ignoring the cross links), the projections are TAGs for an English fragment and an LF fragment, respectively. These grammars are themselves written in a particular variant of TAGs; the choice of this base formalism, as we will call it, is free. In the case at hand, we have chosen single-component lexicalized TAGs with adjunction and substitution (Schabes, Abeill6 and Joshi \[1988\]). Other bases (as Multiple Component TAGs) are needed for more complex phenomena.</Paragraph>
    <Paragraph position="10"> The elementary operation in a synchronous TAG is supervenient on the elementary operations in the base formalism.</Paragraph>
    <Paragraph position="11"> A derivation step from a pair of trees (oq, o~2) proceeds as follows: 1. Nondeterministically choose a link in the pair connecting two nodes (say, nl in cq and n~ in c~2).</Paragraph>
    <Paragraph position="13"> 2. Nondeterministically choose a pair of trees (#x, f12) in the grammar.</Paragraph>
    <Paragraph position="14"> 3. Form the resultant pair (fll(Otl, nl), fl2(ol2, n2)) where  ~(o~, n) is the result of performing a primitive operation in the base formalism on o~ at node n using ~/ (e.g., adjoining or substituting B into ot at n). 1 Synchronous TAG derivation then proceeds by choosing a pair of initial trees (oq, o~2) that is an element of the grammar, and repeatedly applying derivation steps as above. As an example, suppose we start with the tree pair o~ in Figure 1. 2 We choose the link from the subject NP to T and the tree pair ~ to apply to its nodes. The resultant, by synchronous substitution, is the tree pair:</Paragraph>
    <Paragraph position="16"> c~l except for the chosen link, which has no counterpart in the result.</Paragraph>
    <Paragraph position="17"> Using tree pair 7 on the remaining link from NP to T in  ot 2 I NP VP ~ R T ~ T /I I I 1&amp;quot;I k Gedegrge y ? hate'gedegrgeTbrdegccdegli&amp;quot;  This pairing manifests the correspondence between the sentence &amp;quot;George hates broccoli&amp;quot; and its logical form hates'(george', broccoli ~) (as written in a more traditional notation). Here we see that the links in the operator trees (those in 7) are preserved in the resultant pair, accounting for the sole remaining link. The trees in 7 are linked in this way so that other tree pairs can modify the N.</Paragraph>
    <Paragraph position="18"> We can continue the derivation, using 8 and e to generate the pair given in Figure 2 thereby associating the meaning violently' ( hates' (george', cooked' (broccoli') ) ) ) with the sentence &amp;quot;George hates cooked broccoli violently.&amp;quot; The arguments for factoring recursion and dependencies as TAGs do for the syntax of natural language have their counterparts in the semantics. The structure of TAGs allows syntactic dependencies--agreement, subcategorization, and so forth--to be localized in the primitives of a grammar, the elementary trees. This is most dramatically evident in the case of long-distance dependencies, such as that between a wh-phrase and its associated gap. Similarly, using TAGs to construct logical forms allows the localization of semantic dependencies in the logical forms of natural language expressions, dependencies such as the signature requirements (argument type and arity) of function and relation symbols, and even the long-distance dependencies between a wh-quantifier and its associated bound variable. With other methods of semantics, these dependencies cannot be localized; the semantic aspects of filler-gap dependencies must be passed among the features of various nodes in a parse tree or otherwise distributed over the entire derivation.</Paragraph>
    <Paragraph position="19"> The use of the synchronous TAG augmentation allows an even more radical reduction in the role of features in a TAG grammar. Because of the extended domain of locality that TAGs possess, the role of features and unification is reduced from its role in context-free based systems. Only finite-valued features are needed, with the possible exception of a feature whose value encodes an expression's logical form. In removing the construction of logical forms from the duties delegated to features, we can maintain a strictly finite-valued---and therefore formally dispensable--feature system for TAGs.</Paragraph>
    <Paragraph position="20">  Synchronous TAGs suggest elegant solutions to the semantics of idioms, quantifier scoping (Shieber and Schabes, \[1990a\]) and provide an elegant framework for generation (Shieber and Schabes, \[1990b\]) and machine translation (Abeill6, Schabes and Joshi \[1990\]).</Paragraph>
    <Section position="1" start_page="49" end_page="49" type="sub_section">
      <SectionTitle>
Semantics Idioms
</SectionTitle>
      <Paragraph position="0"> All of the arguments for the TAG analysis of idioms and light verb constructions (Abeill6 and Schabes, 1989) can then be maintained in a formalism that allows for semantics for them as well. In particular, discontinuous syntactic constituents can be semantically localized nonstandard long-distance dependencies are statable without resort to reanalysis, both frozen and flexible idioms can be easily characterized.</Paragraph>
      <Paragraph position="1"> For example, the idiomatic construction &amp;quot;kick the bucket&amp;quot; cashes out as the following tree pair, under its idiomatic interpretation:</Paragraph>
      <Paragraph position="3"> whereas the literal usage of &amp;quot;kick&amp;quot; is associated with a tree pair similar to that of &amp;quot;hates&amp;quot; in Figure 1.</Paragraph>
      <Paragraph position="4"> Quantifiers In order to characterize quantifier scoping possibilities, multi-component TAGs (as defined by Joshi, 1987) is used as the base formalism for synchronous TAG (see Shieber and Schabes \[1990(a)\] for more details on quantifiers scoping with Synchronous TAG). In particular, an NP will be linked both to a formula in the semantics (the quantifier's scope) and a term (the position bound by the quantifier).</Paragraph>
      <Paragraph position="5"> Generation The nondirectionaly of Synchronous TAGs enables us to use it for semantic interpretation as well as for generation (see Shieber and Schabes \[1990b\]).</Paragraph>
    </Section>
    <Section position="2" start_page="49" end_page="49" type="sub_section">
      <SectionTitle>
Machine Translation
</SectionTitle>
      <Paragraph position="0"> The transfer between two languages, such as French and English, can be done by putting directly into correspondence large elementary units without going through some interlingual representation and without major changes to the source and target grammars (Abeill6, Schabes and Joshi \[1990\]).</Paragraph>
      <Paragraph position="1"> The underlying formalism for the transfer is Synchronous Tree Adjoining Grammars. Transfer rules are stated as correspondences between nodes of trees of large domain of locality which are associated with words. We can thus define lexical transfer rules that avoid the defects of a mere word-to-word approach but still benefit from the simplicity and elegance of a lexical approach (this work has been done in collaboration with Anne Abeill6).</Paragraph>
      <Paragraph position="2"> As an example, consider the fragment of the transfer lexicon given in Figure 3.</Paragraph>
      <Paragraph position="3">  For example, suppose we start with the pair 3' and we operate the pair a on the link from the English node NPo to the French node NP1. This operation yields the derived pair a4.</Paragraph>
      <Paragraph position="4">  Then, ff the pair fl operates on the NP1-NPo in 0/4, the following pair 0/5 is generated.</Paragraph>
      <Paragraph position="6"> Finally, when the pak ~ operates on the S-S link in 0/5, the pair 0/6 is generated.</Paragraph>
      <Paragraph position="8"> The fragment of the transfer lexicon given in Figure 3 therefore enables us to translate: Apparently, John misses Mary Apparemment, Mary manque ~ John In most cases, translation can be performed incrementally as the input string is being parsed.</Paragraph>
      <Paragraph position="9"> By virtue of their extended domain of locality, Tree Adjoining Grammars allow regular correspondences between larger structures to be stated without a mediating interlingual representation. The mapping of derivation trees from source to target languages, using the formalism of synchronous TAGs, makes possible to state such direct correspondences. By doing so, we are able to match linguistic units with quite different internal structures. Furthermore, the fact that the grammars are lexicalized enables capturing some idiosyncrasies of each language.</Paragraph>
      <Paragraph position="10"> The simplicity and effectiveness of the transfer rules in this approach shows that lexicalized TAGs, with their extended domain of locality, are very well adapted to machine translation.</Paragraph>
    </Section>
    <Section position="3" start_page="49" end_page="49" type="sub_section">
      <SectionTitle>
Efficient Processing of TAGs
</SectionTitle>
      <Paragraph position="0"> The second development is the design of LR-style parsers for TAGs. LR parsing strategies evolved out of the original work of Knuth. LR(k) parsers for Context Free Grammars (Knuth, 1965) consist of a finite state control (constructed given a CFG) that drives deterministically with k lookahead symbols a push down stack, while scanning the input from left to right. It has been shown that they recognize exactly the set of languages recognized by deterministic push down automata. LR(k) parsers for CFGs have been proven useful for compilers as well as recently for natural language processing. For natural language processing, although LR(k) parsers are not powerful enough, conflicts between multiple choices are solved by pseudo-parallelism (Lang, 1974, Tomita, 1987). This gives rise to a class of powerful yet efficient parsers for natural languages. It is in this context that deterministic (LR(k)-style) parsing of TAGs is studied (this work has been done in collaboration with Vijay-Shanker).</Paragraph>
      <Paragraph position="1"> The set of Tree Adjoining Languages is a strict superset of the set of Context Free Languages (CFLs). For example, the cross serial dependency construction in Dutch can be generated by a TAG. Walters (1970), R6v6sz (1971), Turnbull and Lee (1979) investigated deterministic parsing of the class of context-sensitive languages. However they used Turing machines which recognize languages much more powerful than Tree Adjoining Languages. So far no deterministic bottom-up parser has been proposed for any member of the class of the so-called &amp;quot;mildly context sensitive&amp;quot; formalisms (Joshi, 1985) in which Tree Adjoining Grammars fall. 3 Since the set of Tree Adjoining Languages (TALs) is a strict superset of the set of Context Free Languages, in order to define LR-type parsers for TAGs, we need to use a more powerful configuration then a finite state automaton driving a push down stack. The design of deterministic left to right bottom up parsers for TAGs in which a finite state control drives the moves of a Bottom-up Embedded Push Down Stack has been investigated. The class of corresponding non-deterministic automata recognizes exactly the set of TALs.</Paragraph>
      <Paragraph position="2"> Due to the lack of space, we focus our attention on the bottom-up embedded pushdown automaton. The moves of the parser are sequences of moves of the automaton. The complete construction of LR-style parser for TAGs can be found in Schabes and Vijay-Shanker (1990).</Paragraph>
    </Section>
    <Section position="4" start_page="49" end_page="52" type="sub_section">
      <SectionTitle>
Automata Models of Tags
</SectionTitle>
      <Paragraph position="0"> Before we discuss the Bottom-up Embedded Pushdown Automaton (BEPDA) which is used by parser, we will explain the Embedded Pushdown Automaton (EPDA). An EPDA is similar to a pushdown automaton (IDA) except that the storage of an EPDA is a sequence of pushdown stores. A move of an EPDA (see Figure 5) allows for the introduction of bounded pushdowns above and below the current top pushdown. Informally, this move can be thought of as corresponding to the adjoining operation move in TAGs with the  down reflecting the tree structure to the left and right of the foot node of an auxiliary being adjoined. The spine (path from root to foot node) is left on the previous stack.</Paragraph>
      <Paragraph position="1"> I .,,~left of foot of 13 pme ~.~spine of \[3 H ~'~right degf fdegdegt degf13  The generalization of a PDA to an EPDA whose storage is a sequence of pushdowns captures the generalization of the nature of the derived trees of a CFG to the nature of derived trees of a TAG. From Thatcher (1971), we can observe that the path set of a CFG (i.e. the set of all paths from root to leaves in trees derived by a CFG) is a regular set. On the other hand, the path set of a TAG is a CFL. This follows from the nature of the adjoining operation of TAGs, which suggests stacking along the path from root to a leaf. For example, as we traverse down a path in a tree &amp;quot;r (in Figure 5), if adjunction, say by/3, occurs then the spine of/3 has to be traversed before we can resume the path in &amp;quot;r.</Paragraph>
      <Paragraph position="2"> Bottom-up Embedded Pushdown Automaton For any TAG G, an EPDA can be designed such that its moves correspond to a top-down parse of a string generated by G (EPDA characterizes exactly the set of Tree Adjoining Languages, Vijay- Shanker, 1987). If we wish to design a bottom-up parser, say by adopting a shift reduce parsing strategy, we have to consider the nature of a reduce move of such a parser (i.e. using EPDA storage). This reduce move, for example applied after completely considering an auxiliary tree, must be allowed to 'remove' some bounded pushdowns above and below some (not necessarily bounded) pushdown. Thus (see Figure 4), the reduce move is like the dual of the wrapping move performed by an EPDA.</Paragraph>
      <Paragraph position="3"> Therefore, the Bottom-up Embedded Pushdown Automaton (BEPDA), whose moves are dual of an EPDA, has been introduced. The two moves of a BEPDA are the unwrap move depicted in Figure 4 - which is an inverse of the wrap move of an EPDA - and the introduction of new pushdowns on top of the previous pushdown (push move). In an EPDA, when the top pushdown is emptied, the next pushdown automatically becomes the new top pushdown. The inverse of this step is to allow for the introduction of new pushdowns above the previous top pushdown. These are the two moves allowed in a BEPDA, the various steps in our parsers are sequences of one or more such moves.</Paragraph>
      <Paragraph position="4"> Due to space constraints, we do not show the equivalence between BEPDA and EPDA apart from noting that the moves of the two machines are dual of each other.</Paragraph>
      <Paragraph position="5"> Using the BEPDA, the parser recognizes the derived tree inside out: it extracts recursively the innermost auxiliary tree that has no adjunction performed in it. Schabes and Vijay-Shanker (1990) give a complete explanation of the parser moves and its construction. The accuracy of the parsing table can also be improved by computing lookaheads for TAGs.</Paragraph>
      <Paragraph position="6"> Similar to the work of Lang (1974) and Tomita (1987) extending LR parsers for arbitrary CFGs, the LR parsers for TAGs can be extended to solve by pseudo-parallelism the conflicts of moves.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>