Session 11 
Natural Language III 
Kenneth Ward Church 
AT&T Bell Laboratories 
This session consisted of four papers: two papers discussed the integration of text and graphics in 
natural language generation/explanation, one paper on interpreting speech acts and one paper on the 
TAG (Tree Adjoining Grammar) formalism. 
Allen presented "Using Structural Constraints for Speech Act Interpretation" by Allen and Hinkleman. 
The paper discusses ways to distinguish an indirect speech act such as "Can you pass the salt?" from a 
direct yes-no question. It is important that a speech understanding system be able to distinguish the 
two; given an indirect question such as "Can you tell me how many ships are in the Bering Sea?", the 
system should not repsond with "yes". Hinkleman, in her recently completely thesis, has proposed a 
number of rules such as: "If the sentence contains the adverb 'please,' than the possible interpretation 
is: 'directive-command.' " In the oral presentation, Allen also suggested that there might be some 
intonational cues that could be useful. This is an interesting possibility that we ought to think about. 
Schabes then presented "An Evaluation of Lexicalization in Parsing," by Joshi and Schabes. Over the 
last few years, there has been considerable interest in "head-driven" parsing. That is, the parser would 
somehow take advantage of constraints between a verb (the head of a verb phrase) and its arguments in 
order to parse more quickly and more accurately. Although there is a very strong intuition that some 
form of this argument must be correct, much of the literature has been disappointingly vague. In 
contrast, Joshi and Schabes report on an experiment showing that lexical constraints improves parsing 
time. The experiment used a small grammar fragment written in the TAG formalism. They plan to 
investigate how well these results scale up with larger grammars. It would also be interesting to see 
how well these results generalize over input corpora and over grammatical formalisms. 
The last two papers, "Natural Langauge with Integrated Deictic and Graphic Gestures" by Neal 
(presenter), Theilman, Dobes, Hailer, Galnowski and Shapiro, and "Coordinating Text and Graphics in 
Explanation Generation," by Feiner and McKeown (presenter), discussed a multi-media approach to 
generation. Sometimes a picture is more appropriate than an explanation in English. Both pictures and 
English text have constraints. Text should not be ungrammatical, or very awkward. So, too, pictures 
need to fit in a certain space and shouldn't be too busy. Feiner and McKeown suggest that the 
constraints should be described declaratively with a unification formalism. It might also be interesting 
to consider other forms of multi-media presentation such as speech. 
383 
