A REAL-TIME SPOKEN-LANGUAGE SYSTEM 
FOR INTERACTIVE PROBLEM SOLVING 
Patti Price and Robert C. Moore 
SRI International 
Menlo Park, CA 94025 
1. PROJECT GOALS 
The goal of this project, to develop a spoken language 
interface to the Official Airline Guide (OAG) database, 
has been developed along two overlapping research and 
development lines: one focussed on an SLS kernel for 
database query, and the other on the interactive system. 
2. RECENT RESULTS 
SRI has developed a spoken language system to re- 
trieve air travel planning information. Progress can be 
measured by comparing DARPA benchmark results in 
February 1992 and November 1992. Between February 
1992 and November 1992, for all utterances tested, SRJ's 
word error rate in the ATIS speech recognition test im- 
proved from 11.0% to 9.1%. Weighted utterance error 
improved from 31.1% to 23.6% in the natural-language 
understanding test, and from 45.4% to 33.2% in the 
spoken-language understanding test. Other recent re- 
sults include: 
• Ported ATIS system to new, 46-city database; coor- 
dinated with internally funded effort to port to the 
online OAG. 
• Improved speech understanding by modeling spon- 
taneous speech phenomena, including filled pauses 
and verbal repairs. The study included development 
of method for labeling and classifying repairs and 
tools for their analysis. 
• Developed algorithms for tracking discourse struc- 
ture that correctly determine discourse context with 
greater than 90% accuracy on the ATIS training cor- 
pus. 
• Improved GEMINI system' s finguistic coverage of 
the ATIS task to 93% syntactic coverage and 86% 
semantic coverage in a fair test on the evaluable 
utterances in the November 1992 ATIS benchmark 
test set. 
• Integrated GEMINI system into overall ATIS sys- 
tem, and used it for the benchmark tests in con- 
junction with the Template Matcher. 
* Performed pilot experiments on the use of natural 
language constraints to improve speech recognition; 
initial evidence showed an encouraging 22% reduc- 
tion in word error rate. 
• Analyzed human-machine problem solving using 
SRI's ATIS system. Analyzed user satisfaction and 
system performance as a function of system errors 
and user experience. 
• Collected ATIS training and test data (speech, tran- 
scriptions, and logfiles) using SRI ATIS system, in- 
cluding about a thousand utterances using the new 
database. 
• Improved user interface ofSRI ATIS system, includ- 
ing better paraphrasing of system's understanding, 
easier to read displays, and improved system error 
messages. 
• Produced one journal publication, sixteen confer- 
ence talks and proceedings papers, and many in- 
vited talks; provided support and training for four 
graduate students. 
3. PLANS 
Complete integration of GEMINI system into SRI 
ATIS, including tighter integration with Template 
Matcher. 
Explore tighter integration of speech and NL pro- 
eessing for better overall SLS performance, perhaps 
via lattices. 
Improve system robustness, portability and scala- 
bility. 
Develop complete telephone-based ATIS system. 
414 
