SESSION 5: NATURAL LANGUAGE, DISCOURSE 
Paul S. Jacobs, Chair 
Information Technology Laboratory 
GE Research and Development Center 
Schenectady, NY 12301 
OVERVIEW AND BACKGROUND 
The papers in this group cover a broad range of topics 
from different perspectives. They have in common an 
emphasis on the handling of frequently-occurring phe- 
nomena in real data sets of spoken and written language, 
phenomena which are in some sense outside of the scope 
of some of the core problems in human language tech- 
nologies. We can view problems such as acoustic process- 
ing, word recognition, sentence parsing, and word sense 
disambiguation as core problems because they have a 
wealth of published literature and a set of broadly ap- 
plied techniques. By contrast, the natural language pa- 
pers in this session hit upon issues like recognizing speech 
repairs and designing "templates" to capture informa- 
tion and test text understanding. These issues are also 
central to HLT work, but have certainly not evolved into 
mature practices. 
Within this general framework, three of the papers ad- 
dress written language work, including template design 
and processing methods, while the remaining two pa- 
pers are on spoken language work, including repairs and 
intonation. 
The two speech papers represent innovative work done 
independently in universities, while the three written 
language papers are part of a community effort to de- 
velop and evaluate systems for extracting data from 
free text, and are all thus all directly related to the 
ARPA-funded TIPSTER program. The two speech pa- 
pers, therefore, are quite self-standing. However, to ap- 
preciate the three written language papers, it helps to 
have a general understanding of TIPSTER data extrac- 
tion and why, for example, some of the themes of these 
papers, such as template design and linguistic motiva- 
tion, are important. Therefore, we will first consider the 
three written language papers here, then turn to the two 
speech papers. 
TIPSTER DATA EXTRACTION 
The TIPSTER (Phase 1) data extraction project, span- 
ning roughly a two-year period from 1991-1993, aimed 
to expand the state of the art in coverage and accu- 
racy in data extractior\[ techniques--systems that derive 
structured information from free text, to support "down- 
stream" text applications such as database retrieval, 
trend analysis, question answering, and so on. TIPSTER 
Phase 2, which focuses on a common, sharable architec- 
ture for such systems along with continuing algorithm 
development, is just beginning. 
In TIPSTER Phase 1, data extraction systems showed 
the ability to cover broad ranges of text in two domains 
(joint ventures and microelectronics) and two languages 
(English and Japanese), with accuracy comparable to 
how similar programs had done a year earlier on much 
easier tasks in a single language and domain. The pro- 
gram thus made considerable progress in scale-up and 
portability, which were its key goals. In addition, the 
TIPSTER-sponsored evaluations, including the recent 
MUC-5 message understanding conference, provided a 
testbed in which many other sites were able to partici- 
pate and compare approaches. 
The papers by Boyan Onyshkevych, "Issues and Method- 
ology for Template Design for Information Extraction", 
and by Jerry Hobbs and David Israel, "Principles of 
Template Design", discuss the infrastructure within 
which work in data extraction is conducted. The data 
extraction task generally uses a corpus of texts, a tem- 
plate design reflecting both the domain of knowledge and 
the information to be extracted, and some examples of 
correctly-filled templates for training. Two particularly 
important criteria of template design help to make such 
work successful: (1) the design must be expressive and 
general enough so that a system that can do a good job 
at filling a particular template can be used successfully 
in new, real applications, and (2) the design must be 
both rich and intuitive, so that the task reveals interest- 
ing characteristics of different methods without making 
a lot of extra work for researchers. The infrastructure 
issues presented in these papers, therefore, are essential 
for progress in the field. 
The paper by Damaris Aynso and the BBN PLUM Re- 
search Group, "Pattern-Matching in a Linguistically Mo- 
tivated Text-Understanding System" summarizes some 
169 
of BBN's efforts in TIPSTER, but also gives a good per- 
spective on some of the results of ARPA-sponsored re- 
search in data extraction. As the paper points out, some 
of the most successful efforts in data extraction have 
gradually dispensed with traditional linguistic knowl- 
edge (such as the largest and most powerful gram- 
mars) and relied more heavily on pattern matching and 
lexically-organized knowledge. This trend has been go- 
ing oll for several years and has been reported, partic- 
ularly by GE and SRI, in previous ARPA workshops. 
Lexically-oriented pattern matching is particularly good 
at quickly capturing the domain and corpus knowledge 
that is required for data extraction. The BBN paper 
suggests that pattern matching and linguistic knowledge 
work best together, a claim that would seem to be sup- 
ported by the fact that systems like SRI's and GE's both 
use pattern matching as an approximation to more pow- 
erful analysis. BBN's system, by contrast, still includes 
a more traditional grammatical component. 
SPEECH REPAIRS AND 
INTONATION 
"Tagging Speech Repairs" by Peter Heeman and James 
Allen, addresses a critical issue in processing real spo- 
ken dialogues--that recognizing and correcting repairs, 
which are frequent in real speech, is necessary to pro- 
cess and understand many spoken inputs. The novelty 
of this work is that it relies heavily on a part-of-speech 
tagger, combining a variety of cues to spot and correct 
the repairs. The results reported, for both recogniti?n 
and correction, are quite good. However, the discus- 
sion of the paper at the HLT meeting did raise some 
of the problems with comparing results of different sys- 
tems and approaches on different data sets; for example, 
related work by Nakatani and Hirschberg used acoustic 
information only and tested only on examples that in- 
cluded repairs, and did only recognition, not correction. 
This raises the question of how these different approaches 
could be effectively compared and even combined. 
"Information Based Intonation Synthesis" by Scott 
Prevost and Mark Steedman uses a rich linguistic 
model to account for problems with contrastive stress 
in dialogues--cases where the simple traditional rule, 
that previously-mentioned word are de-accented, breaks 
down. Although the results of this work so far are more 
limited and anecdotal than the other papers in this ses- 
sion, the approach shows promise for covering a broader 
range of examples of stress and intonation. 
170 
