Dialogue Systems that can Handle Face-to-Face Joint                        
Reference to Actions in Space 
 
 
Justine Cassell 
Dept. of Communication Studies 
Northwestern University 
Evanston, IL 60208 
justine@northwestern.edu
 
 
Abstract 
This talk introduces new research that works towards an 
overarching model of natural face-to-face conversation 
about spatially-located actions in the world, and then 
uses that model to implement a trustworthy embodied  
conversational agent to guide users' ongoing, real-world 
activities away  from the desktop. 
 
Past research has demonstrated that the relationship 
between verbal and nonverbal behavior exists at the 
level of intonational phrases,  conversational turns, dis-
course units, and the negotiation of reference to  objects 
and actions, that mental representations of shared space  
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
are structured in such a way as to allow participants in a 
dialogue to draw on  them, that dialogue systems must 
be based on models of coordination and collaboration, 
and that users are willing to engage in persistent, natu-
ral, trusting conversation with embodied conversational 
systems.  In this talk, these diverse strands of research 
are brought together in the service of a single underly-
ing modality-independent model of action and language, 
non-verbal behaviors and words, production and com-
prehension that can lead to a physically-located, spa-
tially-aware, collaborative embodied conversational 
agent. 
 
 
