Automated Tutoring Dialogues for Training in Shipboard
Damage Control
John Fry, Matt Ginzton, Stanley Peters, Brady Clark& Heather Pon-Barry
Stanford University
Center for the Study of Language Information
Stanford CA 94305-4115 USA
{fry,mginzton,peters,bzack,ponbarry}@csli.stanford.edu
Abstract
This paper describes an application
of state-of-the-art spoken language
technology (OAA/Gemini/Nuance)
to a new problem domain: engaging
students in automated tutorial dia-
logues in order to evaluate and im-
prove their performance in a train-
ing simulator.
1 Introduction
Shipboarddamagecontrolreferstothetaskof
containing the eﬀects of ﬁre, explosions, hull
breaches, ﬂooding, and other critical events
that can occur aboard Naval vessels. The
high-stakes,high-stressnatureofthistask,to-
gether with limited opportunities for real-life
training, make damage control an ideal target
for AI-enabled educational technologies like
training simulators and tutoring systems.
This paper describes the spoken dialogue
systemwedevelopedforautomated critiquing
of student performance on a damage control
training simulator. The simulator is DC-
Train (Bulitko and Wilkins, 1999), an im-
mersive, multimedia training environment for
damage control. DC-Train’s training sce-
nariossimulateamixtureofphysicalphenom-
ena (e.g., ﬁre, ﬂooding) and personnel issues
(e.g., casualties, communications, standard-
ized procedures). Our current tutoring sys-
tem is restricted ﬁre damage scenarios only,
and in particular to the twelve ﬁre scenar-
ios available in DC-Train version 2.5, but
in future versions we plan to support post-
session critiques for all of the damage phe-
nomena that will be modeled by DC-Train
4.0: ﬁre, ﬂooding, missile damage, and wall
or ﬁremain ruptures.
2 Previous Work
Eliciting self-explanation from a student has
been shown to be a highly eﬀective tutoring
method (Chi et al., 1994). For this reason,
a number of automated tutoring systems cur-
rently useNLP techniques to engage students
in reﬂective dialogues. Three notable exam-
ples are the medical Circsim tutor (Zhou et
al., 1999); the Basic Electricity and Electron-
ics (BE&E) tutor (Ros´e et al., 1999); and
thecomputerliteracyAutoTutor(Wiemer-
Hastings et al., 1999).
Our system shares several features with
these three tutoring systems:
A knowledge base Our system encodes
all domain knowledge relevant to supporting
intelligent tutoring feedback into a structure
called an Expert Session Summary (Section
4). These expert summaries encode causal
relationships between events on the ship as
well as the proper and improper responses to
shipboard crises.
Tutoring strategies In our system, as in
those above, the ﬂow of dialogue is controlled
by (essentially) a ﬁnite-state transition net-
work (Fig. 1).
An interpretation component In our
system, thestudent’sspeechisrecognizedand
parsed into logical forms (Section 3). A dia-
logue manager inspects the current dialogue
information state to determine how best to
incorporate each new utterance into the dia-
logue (Lemon et al., 2001).
Prompt
student review
of actions
Correct
student’s
report
Prompt for
reflection on
START
END
continue"
"OK, let’s
event N...
Summary
of damage
main points
Review
performance
student’s
Evaluate
reflections
Correct
student’s
"You handled
this one well"
event 1
of damage
Summary
Brief
summary of
session
errors
Figure 1: Post-session dialogue move graph (simpliﬁed)
However, an important diﬀerence is that
the three systems above are entirely text-
based, whereas ours is a spoken dialogue sys-
tem. Ourspeech interface oﬀers greater natu-
ralnessthankeyboard-basedinput. Inthisre-
spect,oursystemissimilartocove(Roberts,
2000), a training simulator for conning Navy
ships that uses speech to interact with the
student. Butwhereascoveusesshortconver-
sational exchanges to coach the student dur-
ing the simulation, our system engages in ex-
tended tutorial dialogues after the simulation
has ended. Besides being more natural, spo-
ken language systemsare also better suitedto
multimodal interactions (viz., one can point
and click while talking but not while typing).
An additional signiﬁcant diﬀerence between
our system and a number of other automated
tutoring systems is our use of ‘deep’ process-
ing techniques. While other systems utilize
‘shallow’statistical approacheslikeLatentSe-
manticAnalysis(e.g. AutoTutor), oursystem
utilizes Gemini, a symbolic grammar. This
approach enables us to provide precise and
reliable meaning representations.
3 Implementation
To facilitate the implementation of multi-
modal, mixed-initiative tutoring interactions,
we decided to implement our system within
the Open Agent Architecture (OAA) (Martin
et al., 1999). OAA is a framework for coor-
dinating multiple asynchronous communicat-
ing processes. The core of OAA is a ‘facilita-
tor’ which manages message passing between
a number of software agents that specialize
in certain tasks (e.g., speech recognition or
database queries). Our system uses OAA to
coordinate the following ﬁve agents:
1. The Gemini NLP system (Dowding et
al., 1993). Gemini uses a single uniﬁ-
cation grammar both for parsing strings
of words into logical forms (LFs) and for
generating sentences from LF inputs.
2. A Nuance speech recognition server,
which converts spoken utterances to
strings of words. The Nuance server re-
lies on a language model, which is com-
piled directly from the Gemini grammar,
ensuring that every recognized utterance
is assigned an LF.
3. The Festival text-to-speech system,
which ‘speaks’ word strings generated by
Gemini.
4. A Dialogue Manager which coordi-
nates inputsfromtheuser, interprets the
user’s dialogue moves, updates the dia-
logue context, and delivers speech and
graphical outputs to the user.
5. A Critique Planner, described below
in Section 4.
Agents 1-3 are reusable, ‘oﬀ-the-shelf’ dia-
logue system components (apart from the
Gemini grammar, which mustbe modiﬁedfor
each application). We implemented agents 4
and 5 in Java speciﬁcally for this application.
Variants of this OAA/Gemini/Nuance ar-
chitecture have been deployed successfully in
other dialogue systems, notably SRI’s Com-
mandTalk (Stent et al., 1999) and an un-
Figure 2: Screen shot of post-session tutorial dialogue system
manned helicopter interface developed in our
laboratory (Lemon et al., 2001).
4 Planning the dialogue
Each student session with DC-Train pro-
ducesa session transcript, i.e.atime-stamped
record of every event (both computer- and
student-initiated) that occurred during the
simulation. These transcripts serve as the
input to our post-session Critique Planner
(CP).
The CP plans a post-session tutorial di-
alogue in two steps. In the ﬁrst step, an
Expert Session Summary (ESS) is cre-
ated from the session transcript. The ESS
is a tree whose parent nodes represent dam-
age events and whose leaves represent actions
taken in response to those damage events.
Each student-initiated action in the ESS is
evaluatedastoitstimelinessandconformance
to damage control doctrine. Actions that the
studentshouldhavetakenbutdidnotarealso
inserted into the ESS and ﬂagged as such.
Each action node in the ESS therefore falls
into one of three classes: (i) correct actions;
(ii) errors of commission (e.g., the student
sets ﬁre containment boundaries incorrectly);
and (iii) errors of omission (e.g., the student
failstosecurepermissionfromthecaptain be-
fore ﬂooding certain compartments).
Our current tutoring system covers scenar-
ios generated by DC-Train 2.5, which covers
ﬁre scenarios only. Future versions will use
scenarios generated by DC-Train 4.0, which
coversdamagecontrolscenariosinvolvingﬁre,
smoke, ﬂooding, pipe and hull ruptures, and
equipment deactivation. Our current tutor-
ing system is based on an ESS graph that is
generated by an expert model that consists
of an ad-hoc set of ﬁreﬁghting rules. Future
versions will be based on an ESS graph that
is generated by an successor to the Minerva-
DCA expert model (Bulitko and Wilkins,
1999), an extended Petri Net envisionment-
based reasoning system. The new expert
model is designed to produce an ESS graph
duringthecourseofproblemsolvingthatcon-
tains nodes for all successful and unsuccessful
plan and goal achievement events, along with
an explanation structure foreach graph node.
The second step in planning the post-
session tutorial dialogue is to produce a di-
alogue move graph (Fig. 1). This is a di-
rectedgraphthatencodesallpossibleconﬁgu-
rations of dialogue structure and content that
can be handled by the system.
Generating an appropriate dialogue move
graph from an ESS requires pedagogical
knowledge, and in particular a tutoring strat-
egy. The tutoring strategy we adopted is
based on our analysis of videotapes of ﬁfteen
actual DC-Train post-session critiques con-
ducted by instructors at the Navy’s Surface
Warfare Oﬃcer’s School in Newport, RI. The
strategy we observed in these critiques, and
implemented in our system, can be outlined
as follows:
1. Summarize the results of the simulation
(e.g., the ﬁnal condition of the ship).
2. For each majordamage event inthe ESS:
(a) Ask the student to review his ac-
tions, correcting his recollections as
necessary.
(b) Evaluate thecorrectness of each stu-
dent action.
(c) If the student committed errors,
ask him how these could have been
avoided, and evaluate the correct-
ness of his responses.
3. Finally, review each type of error that
arose in step (2c).
A screen shot of the tutoring system in
action is shown in Fig. 2. As soon as a
DC-Train simulation ends, the dialogue sys-
tem starts up and the dialogue manager be-
gins traversing the dialogue move graph. As
the dialogue unfolds, a graphical representa-
tion of the ESS is revealed to the student in
piecemeal fashion as depicted in the top right
frame of Fig. 2.
Acknowledgments
This work is supported by the Depart-
ment of the Navy under research grant
N000140010660, a multidisciplinary univer-
sity researchinitiative onnaturallanguagein-
teraction with intelligent tutoring systems.

References
VV.BulitkoandDC.Wilkins. 1999. Automated
instructor assistant for ship damage control. In
Proceedings of AAAI-99, Orlando, FL, July.
M. T. H. Chi, N. de Leeuw, M. Chiu, and C.
LaVancher. 1994. Eliciting self-explanations
improves understanding. Cognitive Science,
18(3):439–477.
J. Dowding, J. Gawron, D. Appelt, J. Bear, L.
Cherny, R. C. Moore, and D. Moran. 1993.
Gemini: A naturallanguagesystemfor spoken-
language understanding. In Proceedings of the
ARPA Workshop on Human Language Technol-
ogy.
O. Lemon, A. Bracy, A. Gruenstein, and S. Pe-
ters. 2001. A multi-modal dialogue system for
human-robot conversation. In Proceedings of
NAACL 2001.
D. Martin, A. Cheyer, and D. Moran. 1999.
The OpenAgentArchitecture: a frameworkfor
building distributed software systems. Applied
ArtiﬁcialInteligence , 13(1-2).
B. Roberts. 2000. Coaching driving skills in
a shiphandling trainer. In Proceedings of the
AAAI Fall Symposium on Building Dialogue
Systems for TutorialApplications .
C.P.Ros´e,B.DiEugenio,andJ.D.Moore. 1999.
Adialoguebasedtutoringsystemforbasicelec-
tricity and electronics. In S. P. Lajoie and
M. Vivet, editors, ArtiﬁcialInteligence in Ed-
ucation (Proceedings of AIED’99), pages 759–
761. IOS Press, Amsterdam.
A.Stent,J.Dowding,J.Gawron,E.O.Bratt,and
R. C. Moore. 1999. The CommandTalk spoken
dialogue system. In Proceedings of ACL ’99,
pages 183–190, College Park, MD.
P. Wiemer-Hastings, K. Wiemer-Hastings, and
A. Graesser. 1999. Improving an intelli-
gent tutor’s comprehension of students with la-
tent semantic analysis. In S. P. Lajoie and
M. Vivet, editors, ArtiﬁcialInteligence in Ed-
ucation (Proceedings of AIED’99), pages 535–
542. IOS Press, Amsterdam.
Y. Zhou, R. Freedman, M. Glass, J. A. Michael,
A. A. Rovick,and M. W. Evens. 1999. Deliver-
inghintsinadialogue-basedintelligenttutoring
system. In Proceedings of AAAI-99, Orlando,
FL, July.
