Research and Development for Spoken Language Systems 
Victor W. Zue 
Spoken Language Systems Group 
Laboratory for Computer Science 
Massachusetts Institute of Technology 
OBJECTIVE: 
The goal of this research is to develop a spoken language system that will demonstrate the usefulness 
of voice input for interactive problem solving. The system will accept continuous speech, and will handle 
multiple speakers without explicit speaker enrollment. Combining speech recognition and natural language 
processing to achieve speech understanding, the system will be demonstrated in an application domain 
relevant to the DoD. The objective of this project is to develop a robust and high-performance speech 
recognition system using a segment-based approach to phonetic recognition. The recognition system will 
eventually be integrated with natural language processing to achieve spoken language understanding. 
SUMMARY OF ACCOMPLISHMENTS: 
• Developed the VOYAGER system that combines speech recognition SUMMIT with natural language pro- 
cessing TINA to achieve speech understandingwithin the task domain of navigational assistance. The 
system was brought into existence during the first quarter of the contracting period. 
• Collected a spoken language database of approximately 10,000 sentences from 100 speakers using the 
VOYAGER system. Performed comparative acoustic and linguistic analysis of spontaneous and read 
speech. 
• Conducted preliminary evaluation of VOYAGER using the spontaneous speech database. Achieved word 
and sentence recognition accuracy of 86 and 49%, respectively, for sentences that passed the word pair 
grammar. 
PLANS: 
• Improve the speech recognition performance by incorporating context-dependency in phonetic mod- 
elling. 
• Fully integrate TINA and SUMMIT in order to exploit speech and natural language symbiosis. 
• Increase and improve the knowledge base of VOYAGER., SO that correct and natural responses can be 
generated. 
• Collect additional speech and text data during actual problem solving for system development and 
evaluation, and continue to evaluate the performance of VOYAGER. 
• Begin hardware development, such that the system will soon be able to run in near real-time. 
450 
