DEVELOPMENT OF A SPOKEN LANGUAGE SYSTEM 
John Makhoul and Madeleine Bates 
makhoul@bbn.com, bates@bbn.com 
BBN Systems and Technologies 
10 Moulton Street 
Cambridge, MA 02138 
PROJECT GOALS 
The primary objective of this project is to develop a 
robust, high-performance spoken language system. We 
have achieved this by integrating the BYBLOS speech 
recognition system with the DELPHI natural language 
processing system to produce the HARC (Hear And 
Respond to Continuous speech) system. 
RECENT RESULTS 
Achieved a weighted error rate of 43.7 on the 
official ATIS spoken language system (SLS) 
evaluation for the Feburary, 1992 workshop. 
This is the lowest error rate of all systems 
evaluated. 
Achieved a word error rate of 9.4 on the 
official ATIS speech recognition (SPREC) 
evaluation for the February, 1992 workshop. 
This is the lowest error rate of all systems 
evaluated. 
Achieved a weighted error rate of 33.9 on the 
official ATIS natural language (NL) 
evaluation for the February, 1992 workshop. 
This is not significantly different from the 
other top-scoring systems. 
The 1-best output of BBN's speech 
recognition component, paired with SRI's 
natural language component, achieved an 
weighted error rate of 39.9. 
Achieved an unofficial weighted error on the 
Feburary, 1992 test set of 39.23, the lowest 
(though unofficial) rate of all systems we 
currently know of. 
We developed and used a data collection 
facility to collect 2277 utterances from 62 
subjects. We were the only site to reach the 
data collection goals by the original deadline 
of 1 September. 
We completed an initial version of a new 
DELPHI NL system, which incorporates a 
semantic component integrated with the 
unification-based syntactic grammar, but 
separated from it. This allows for multiple 
semantic interpretations for a single parse, 
facilitates rule creation and debugging, and 
allows for both greater expressive power and 
greater computational efficiency. 
We developed a fallback understanding 
component for DELPHI which is composed 
of a fragment generator, a syntactic 
combiner, and a frame combiner. This 
fallback strategy was included in the system 
used in the official evaluation. 
For our real-time decoder work, we sped up 
the decoder so that it can run in real-rime 
with a higher beamwidth, and, therefore, 
higher accuracy. 
We ported the real-time speech system to run 
on the SGI workstation. 
We developed tools for creating and 
estimating various types of statistical 
grammars. 
PLANS FOR THE COMING YEAR 
For the coming year, we plan to continue our work on 
improving both the speech recognition perofrmance and the 
natural language processing performance of the HARC 
system. We will pay particular attention to the discourse 
phenomena that are so important in the ATIS domain, and 
will take increasing advantage of probabilistic analysis at 
several levels of processing. We will in particular 
investigate methods for automatic training of the NL 
component. We will also study more deeply the 
interaction of the speech and NL components via the N- 
best interface. 
463 
