r 
Narrated Animation: A Case for Generation 
Norman Badler Mark Steedman 
Department of Computer and Information Science 
University of Pennsylvania 
Philadelphia PA 19104-63891 
Bonnie Lynn Webber 
1 Introduction 
Our project rests on the belief that computer animation in the form of narrated animated 
simulations can provide an engaging, effective and flexible medium for instructing agents of 
varying capabilities to perform tasks that make varying demands in workplaces of varying layout. 
To this end, we have been designing and implementing an integrated system which combines 
• animated agents which can demonstrate the behavior to be emulated; 
• automatic generation of appropriate Natural Language narration which can explain what 
is being done and why. 
To date, our primary concern with Natural Language has been as input to the system, in 
line with the strong claim we make in \[1\] that moving task animation beyond direct graphical 
manipulation forces one to Natural Language as the only instruction source accessible to other 
users than the current community of manually skilled (or programming-wise) animators. (To 
this end, we have been analysing constructions commonly found in NL instructions, in terms of 
their representational requirements \[3\]. 
However here our point of discussion is NL Generation. What makes us such eager con- 
sumers of advances and technology in this area is that animated simulations without narration 
(ultimately, spoken narration) is only half the story. As researchers studying plan inference 
have shown \[2\], it may be well-nigh impossible to infer an agent's intentions simply by observing 
his or her actions alone. 2 And we know that the ability to perform an action effectively in a 
range of environments requires understanding its intention, not just the physical motions used 
in some performance. Thus, communicating intentions is as important to effective task instruc- 
tion as demonstrating physical skills. Sharing the burden of communication between Natural 
Language and graphics, as Feiner and McKeown have noted \[4\], takes advantage of the best of 
both possible worlds. 
While some parts of our system are further along than others, no work at all has yet been 
done on generation. However, we have tried to take account of the needs of generation in 
designing the system, so that we will not have painted ourselves in a hole from the start. We 
clearly and hope to get further ideas and direction from this meeting. Basically, the system has 
been designed so that the generator will receive information from three sources (see Figure 1.): 
1 This research is partially supported by Lockheed Engineering and Management Services (NASA Johnson 
Space Center), NASA Ames Grant NAG-2-426, FMC Corporation, Martin-Marietta Denver Aerospace, NSF 
CISE Grant CDA88-22719, and ARO Grant DAAL03-89-C-0031 including participation by the U.S. Army Human 
Engineering Laboratory. 
Exaggerating behavior to make it more communicative may have the adverse effect of making it less veridical, 
a situation inversely turned advantageous by skilled cartoon animators \[5\]. 
189 
* the partial global plan (an incrementally computed description of what the animated agent 
is meant to do and why); 
• the basic animation commands (for particulars of what's happening "now") 
• the visualization plan (for what can the viewer see). 
The resulting narrative is thus meant to satisfy the joint communicative goals of providing 
an overall context in which to view the events on the "screen" and explaining the reasons for 
particular events that are happening, thereby transcending the merely visible portion of any 
event, to augment and reinforce observable behavior. For a more detailed description of the 
system and further discussion of instructions and task performance, the reader is referred to \[1\]. 

References 
Norman Badler, Bonnie Webber, Jeff Esakov and Jugal Kalita. Animation from Instruc- 
tions. Making Them Move: Mechanics, Control and Animation of Articulated Figures. 
Morgan-Kaufmann, 1990. (Also appears as Technical Report CIS-90-17, Dept. of Com- 
puter and Information Science, Univ. of Pennsylvania, Philadelphia PA, 1990.) 
Phil Cohen. The Need for Referent Identification as a Planned Action. Proc. of Interna- 
tional Joint Conference on Artificial Intelligence, August 1981, pp. 31-36, 
Bonnie Webber and Barbara Di Eugenio. Free Adjuncts in Natural Language Instructions. 
Proc. of COLING-90. University of Helsinki, Finland. August 1990. 
Feiner, S. and McKeown, K. Coordinating Text and Graphics in Explanation Genera- 
tion. Proc. ARPA Speech and Natural Language Workshop, October 1989, Los Altos CA: 
Morgan Kaufmann, pp. 424-433. 
Frank Thomas and Ollie Johnston. Disney Animation: The Illusion of Life. Abbeville 
Press, New York, 1981. 
