SESSION 14: GOVERNMENT PANEL 
Charles L. Wayne, Chair 
Software and Intelligent Systems Technology Office 
Defense Advanced Research Projects Agency 
Arlington, Virginia 
ABSTRACT 
The Workshop ended with a first-time ever, hour-long 
Government Panel session organized at the request of the 
Workshop Program Committee, which wanted to hear 
from some of the government people present. The panel 
consisted of five individuals, each invited to be provoca- 
tive and each given five minutes to offer his or her per- 
sonal perspectives. The floor was subsequently thrown 
open for a general discussion of the issues raised. 
PANEL STATEMENTS 
Dr. W. T. Chien of the National Science Foundation dis- 
cussed the High Performance Computing and Commu- 
nication (HPCC) initiative, indicating that it presents 
new opportunities for government agencies to coordinate 
their research efforts to address Grand Challenge prob- 
lems, including speech and language. He pointed out 
that these problems require multidisciplinary solutions 
and that HPCC could have a broad and beneficial im- 
pact. 
Dr. John Prange of the National Security Agency indi- 
cated pleasure with the higher visibility being given to 
natural language research. He worried about the simi- 
larity of approaches (variations on Hidden Markov Mod- 
els) being used by many of the speech groups and the 
current methods of evaluation (which may be focusing 
efforts too heavily in directions amenable to evaluation). 
Dr. Prange wants to encourage new ideas and a balance 
between technology and science, while pushing for porta- 
bility, scalability, and language independence. He cited 
TIPSTER as a good example of the benefits of having 
other agencies heavily involved in DARPA projects from 
the start. 
Dr. Susan Chipman of the Office of Naval Research 
spoke from the perspective of a psychologist interested 
both in cognitive science and in applications such as 
training. She wondered about the generalizability of 
DARPA's speech work to other applications and worried 
about using brute force statistics without good theory. 
Admitting some ignorance about how far speech tech- 
nology had come, she noted that in trying out various 
demo systems she was more struck by the failures (e.g., 
nonsensical answers) than the successes. She opined that 
the solution could be to incorporate more top-down con- 
straints as humans do. 
Dr. Tim Anderson of the Armstrong Laboratory ad- 
dressed the Air Force's interest in speech recognition, 
outlining both the challenges and the benefits of putting 
interactive voice in the cockpit. He wants systems that 
need little or no speaker-dependent training, that can 
recognize connected speech under adverse conditions 
with a 100-1000 word vocabulary, and that can talk back 
fluently. Dr. Anderson was particularly pleased and im- 
pressed with the cooperation among sites, but thought 
it unwise for the majority of the community to ride one 
wagon (HMMs). He made a plea for more innovation 
and surmised that many researchers stay on that wagon 
because of the annual evaluation cycles. He also pointed 
out the need to improve portability between application 
domains. He hopes to see interactive voice technology 
become good enough for the Air Force to use in a new 
generation of aircraft. 
Dr. Jordan Cohen of the Institute for Defense Analyses 
claimed that it is more effective to try to solve specific 
problems, then generalize, than to start by looking for 
generic solutions. Therefore, he urged DARPA to find 
real problems and deal with real users who understand 
the real requirements. He hoped that the Air Travel In- 
formation System (ATIS) could become a live database 
(not just a snapshot) with which real users could inter- 
act, and he said that disagreements over the details of 
the new CSR corpus would become mute as soon as we 
find a real customer. Dr. Cohen indicated that he was 
very pleased with what he had seen at the workshop, 
that things had come a long way. 
AUDIENCE REACTIONS 
The ensuing discussion dealt primarily with the issues 
of innovation, evaluation, requirements, and technology 
transfer. 
Responding to the challenge to do more innovation, one 
PI spoke of the need for the Government to help assure 
447 
stronger funding over longer periods of time - a point the 
panel agreed with. A visitor commented that frequent 
evaluations may be encouraging researchers to hug the 
shore rather than to try risky, long term research. 
A senior researcher agreed with the need for innovation, 
but said that deemphasizing evaluation would surely tor- 
pedo progress; he would foster innovation by selecting 
appropriately challenging problems and corpora. A pan- 
elist commented that evaluation currently seems to he 
driving/defining research, whereas it should be the other 
way around, with evaluation methods chosen to shed 
light on the real problems. A government person in the 
audience countered that anyone wishing to use an un- 
conventional approach must have a way to show that it 
is in fact providing benefit. 
Recalling the Sunday afternoon comments by LTC Met- 
tala, who identified validated Army "requirements" for 
speech technology, one person pointed out that there is 
a chicken and egg problem, where one needs to demon- 
strate at least an initial capability before the Services 
can accurately identify their requirements. On the other 
hand, the enunciation of a "requirement" makes it much 
easier to justify spending research dollars to address 
those perceived needs. 
In that vein, it was pointed out that MUC-2 had exposed 
the feasibility of the data extraction application in the 
abstract, that TIPSTER was designed to build on those 
results, and that it is now benefiting from the interaction 
with real users. 
Some researchers argued that it is not their job to do 
technology transfer, that they must first develop some- 
thing to transfer. One claimed that it was not practical 
to focus on customers at this time, while another said 
that it was better to get users involved to pull the tech- 
nology in the right directions. 
It was said that we should not think of speech and 
language technology as end products, but as enabling 
technology which will succeed when it is embedded in a 
larger environment and becomes a regular part of daily 
life. Although there are many important military appli- 
cations for this technnology, the underlying technology 
transcends application domains and has a wide range of 
civilian applications as well. 
The Chair, who is also the Program Manager, closed the 
session by congratulating the researchers on the enor- 
mous progress reported at the Workshop. 
448 
