Research and Development for Spoken Language Systems 
Victor W. Zue 
Spoken Language Systems Group 
Laboratory for Computer Science 
Massachusetts Institute of Technology 
OBJECTIVE: 
The goal of this research is to develop a spoken language system that 
will demonstrate the usefulness of voice input for interactive problem solving. The system 
will accept continuous speech, and will handle multiple speakers without explicit speaker enroll- 
ment. Combining SUMMrr, a segment-based speech recognition system, and TINA, a probabilistic 
natural language system, to achieve speech understanding, the system will be demonstrated in an 
application domain relevant to the DoD. 
SUMMARY OF ACCOMPLISHMENTS: 
• Improved the performance and expanded the capabilities of the VOYAGER urban exploration 
and navigation system. Specifically: 
- incorporated an N-best interface between speech recognition and natural language processing, 
- expanded the coverage of the natural language component and the back-end, and 
- ported VOYAGER tO a Sun workstation with a set of commercially available DSP boards, 
pipelined the computation, and reduced the computation time to approximately 5 times 
real-time. 
Developed a mechanism for automatically generating tasks in the VOYAGER framework to 
promote interactive problem solving by users, thus enabling us to collect spontaneous speech 
from users in a goal-directed mode, 
Performed acoustic and linguistic analysis on nearly 3,000 sentences, contrasting the differ- 
ences between read and spontaneous speech. 
Developed the initial version of ATIS, collected pilot data, and participated in the first round 
of common evaluation. 
PLANS: 
• Improve the speech recognition performance by incorporating context-dependency in pho- 
netic modelling. 
• Fully integrate TINA and SUMMIT in order to exploit speech and natural language symbiosis. 
• Continue to increase and improve the knowledge base of VOYAGER, SO that correct and natural 
responses can be generated. 
• Collect additional speech and text data dunng actual problem solving for system development 
and evaluation, and continue to evaluate the performance of VOYAGER. 
• Port SUMMIT tO the ATIS task, and perform overall system evaluation. 
• Continue hardware development, such that the system will soon be able to run in near 
real-time. 
418 
