<?xml version="1.0" standalone="yes"?>
<Paper uid="P91-1005">
  <Title>AN ALGORITHM FOR PLAN RECOGNITION IN COLLABORATIVE DISCOURSE*</Title>
  <Section position="4" start_page="0" end_page="33" type="metho">
    <SectionTitle>
ACTION P~EPRESENTATION
</SectionTitle>
    <Paragraph position="0"> We use the action representation formally defined by Balkanski (1990) for modelling collaborative actions.</Paragraph>
    <Paragraph position="1"> We use the term act-type to refer to a type of action; e.g. boiling water is an act-type that will be represented by boil(water). In addition to types of actions, we also need to refer to the agents who will perform those actions and the time interval over which they will do so. We use the term activity to refer to this type of information1; e.g. Carol's boiling water over some time interval (tl) is an activity that will be represented by (boil(water),carol,tl). Throughout the rest of this paper, we will follow the convention of denoting arbitrary activities using uppercase Greek letters, while using lowercase Greek letters to denote act-types. In 1This terminology supersedes that used in (Lochbaum et al., 1990).</Paragraph>
    <Paragraph position="2">  addition, lowercase letters denote the act-type of the activity represented by the corresponding uppercase letter, e.g. 7 -- act-type(F).</Paragraph>
    <Paragraph position="3"> Balkanski also defines act-type and activity constructors and relations; e.g. sequence(boil(water), add(noodles,water)) represents the sequence of doing an act of type boil(water) followed by an act of type add(noodles,water), while CGEN(mix(sauce,noodles), make(pasta_dish),C) represents that the first act-type conditionally generates the second (Goldman, 1970; Pollack, 1986). Table 1 lists the act-type and corresponding activity relations and constructors that will be used in this paper.</Paragraph>
    <Paragraph position="4"> Act-type constructors and relations are used in specifying recipes. Following Pollack (1990), we use the term recipe to refer to what an agent knows when the agent knows a way of doing something.</Paragraph>
    <Paragraph position="5"> As an example, a particular agent's recipe for lifting a piano might be CGEN(simult(lift(foot(piano)), lift(keyboard(piano))), lift(piano), AG.\[IGI=2\]); this recipe encodes that simultaneously lifting the foot- and keyboard ends of a piano results in lifting the piano, provided that there are two agents doing the lifting.</Paragraph>
    <Paragraph position="6"> For ease of presentation, we will sometimes represent recipes graphicMly using different types of arrows to represent specific act-type relations and constructors. Figure 1 contains the graphical presentation of the piano lifting recipe.</Paragraph>
    <Paragraph position="8"/>
  </Section>
  <Section position="5" start_page="33" end_page="33" type="metho">
    <SectionTitle>
THE SHAREDPLAN AUGMENTATION
ALGORITHM
</SectionTitle>
    <Paragraph position="0"> A previous paper (Lochbaum et hi., 1990) describes an augmentation algorithm based on Grosz and Sidner's SharedPlan model of collaboration (Grosz and Sidner, 1990) that delineates the ways in which an agent's beliefs are affected by utterances made in the context of collaboration. A portion of that algorithm is repeated in Figure 2. In the discussion that follows, we will assume the context specified by the algorithm.</Paragraph>
    <Paragraph position="1">  SharedPlan*(G1,G2,A,T1,T2) represents that G1 and G2 have a partial SharedPlan at time T1 to perform act-type A at time T2 (Grosz and Sidner, 1990).</Paragraph>
    <Paragraph position="2">  Assume: Act is an action of type 7, G~ designates the agent who communicates Prop(Act), Gj designates the agent being modelled i, j E {1,2}, i ~ j, SharedPlan*(G1 ,G~,A,T1,T2).</Paragraph>
    <Paragraph position="3">  4. Search own beliefs for Contributes(7,A) and where possible, more specific information as to how 7 contributes to A.</Paragraph>
    <Paragraph position="4">  Step (4) of this algorithm is closely related to the standard plan recognition problem. In this step, agent Gj is trying to determine why agent G~ has mentioned an act of type 7, i.e. Gj is trying to identify the role Gi believes 7 will play in their SharedPlan. In our previous work, we did not specify the details of how this reasoning was modelled. In this paper, we present an algorithm that does so. The algorithm uses a new construct: augmented rgraphs.</Paragraph>
  </Section>
  <Section position="6" start_page="33" end_page="34" type="metho">
    <SectionTitle>
AUGMENTED RGRAPH CONSTRUCTION
</SectionTitle>
    <Paragraph position="0"> Agents Gi and Gj each bring to their collaboration private beliefs about how to perform types of actions, i.e.</Paragraph>
    <Paragraph position="1"> recipes for those actions. As they collaborate, a significant portion of their communication is concerned with deciding upon the types of actions that need to be performed and how those actions are related. Thus, they establish mutual belief in a recipe for action s. In addition, however, the agents must also determine which 2Agents do not necessarily discuss actions in a fixed order (e.g. the order in which they appear in a recipe). Consequently, our algorithm is not constrained to reasoning about actions in a fixed order.</Paragraph>
    <Paragraph position="2">  agents will perform each action and the time interval over which they will do so, in accordance with the agency and timing constraints specified by their evolving jointly-held recipe. To model an agent's reasoning in this collaborative situation, we introduce a dynamic representation called an augmented recipe graph. The construction of an augmented recipe graph corresponds to the reasoning that an agent performs to determine whether or not the performance of a particular activity makes sense in terms of the agent's recipes and the evolving SharedPlan.</Paragraph>
    <Paragraph position="3"> Augmented recipe graphs are comprised of two parts, a recipe graph or rgraph, representing activities and relations among them, and a set of constraints, representing conditions on the agents and times of those activities. An rgraph corresponds to a particular specification of a recipe. Whereas a recipe represents information about the performance, in the abstract, of act-types, an rgraph represents more specialized information by including act-type performance agents and times. An rgraph is a tree-like representation comprised of (1) nodes, representing activities and (2) links between nodes, representing activity relations.</Paragraph>
    <Paragraph position="4"> The structure of an rgraph mirrors the structure of the recipe to which it corresponds: each activity and activity relation in an rgraph is derived from the corresponding act-type and act-type relation in its associated recipe, based on the correspondences in Table 1. Because the constructors and relations used in specifying recipes may impose agency and timing constraints on the successful performance of act-types, the rgraph representation is augmented by a set of constraints.</Paragraph>
    <Paragraph position="5"> Following Kautz, we will use the term explaining to refer to the process of creating an augmented rgraph.</Paragraph>
  </Section>
  <Section position="7" start_page="34" end_page="35" type="metho">
    <SectionTitle>
AUGMENTED RGRAPH SCHEMAS
</SectionTitle>
    <Paragraph position="0"> To describe the explanation process, we will assume that agents Gi and Gj are collaborating to achieve an act-type A and Gi communicates a proposition from which an activity F can be derived 3 (cf. the assumptions of Figure 2). Gj's reasoning in this context is modelled by building an augmented rgraph that explains how F might be related to A. This representation is constructed by searching each of Gj's recipes for A to find a sequence of relations and constructors linking 7 to A. Augmented rgraphs are constructed during this search by creating appropriate nodes and links as each act-type and relation in a recipe is encountered.</Paragraph>
    <Paragraph position="1"> By considering each type of relation and constructor that may appear in a recipe, we can specify general schemas expressing the form that the corresponding augmented rgraph must take. Table 2 contains the schemas for each of the act-type relations and  The algorithm for explaining an activity F according to a particular recipe for A thus consists of considering in turn each relation and constructor in the recipe linking 7 and A and using the appropriate schema to incrementally build an augmented rgraph.. Each schema specifies an rgraph portion to create and the constraints to associate with that rgraph. If agent G/ knows multiple recipes for A, then the algorithm attempts to create an augmented rgraph from each recipe. Those augmented rgraphs that are successfully created are maintained as possible explanations for F until more information becomes available; they represent Gj's current beliefs about Gi's possible beliefs. If at any time the set of constraints associated with an augmented rgraph becomes unsatisfiable, a failure occurs: the constraints stipulated by the recipe are not met by the activities in the corresponding rgraph. This failure corresponds to a discrepancy between agent Gj's beliefs and those Gj has attributed to agent G~.</Paragraph>
    <Paragraph position="2"> On the basis of such a discrepancy, agent G i might query Gi, or might first consider the other recipes that she knows for A (i.e. in an attempt to produce a successful explanation using another recipe). The algorithm follows the latter course of action. When a recipe does not provide an explanation for F, it is eliminated from consideration and the algorithm continues looking for &amp;quot;valid&amp;quot; recipes.</Paragraph>
    <Paragraph position="3"> To illustrate the algorithm, we will consider the reasoning done by agent Pare in the dialogue in Figure 3; we assume that Pam knows the recipe given in Figure 1. To begin, we consider the activity derived from utterance (3) of this discourse: F1 =(lift(foot(piano)), {joe},tl), where tl is the time interval over which the agents will lift the piano. To explain F1, the algorithm creates the augmented rgraph shown in Figure 4. It begins by considering the other act-types in the recipe to which 7x=lift(foot(piano))is related. Because 71 is a component of a simultaneous act-type, the simult schema is used to create nodes N1, N2, and the link between them. A constraint of this schema is that the constituents of the complex activity represented by node N2 have the same time. This constraint is modelled directly in the rgraph by creating the activity corresponding to lift(keyboard(piano)) to have the same time as F1. No information about the agent of this activity is known, however, so a variable, G1, is used to represent the agent. Next, because the simultaneous act-type is related by a CGEN relation to lift(piano), the CGEN schema is used to create node N3 and the link between N2 and N3. The first two constraints of the schema are satisfied by creating node N3 such that its activity's agent and time are the  detailed discussion of the derivation of these schemas from the definitions given by Balkanski (1990).</Paragraph>
    <Section position="1" start_page="35" end_page="35" type="sub_section">
      <SectionTitle>
Recipe Augmented Rgraph
Rgraph Constraints
</SectionTitle>
      <Paragraph position="0"/>
      <Paragraph position="2"> same as node N2's. The third constraint is instantiated and associated with the rgraph.</Paragraph>
      <Paragraph position="3">  (1) Joe: I want to lift the piano. (2) Pare: OK.</Paragraph>
      <Paragraph position="4"> (3) Joe: On the count of three, I'll pick up this \[deictic to foot\] end, (4) and you pick up that \[deictic to keyboard\] end.</Paragraph>
      <Paragraph position="5"> (5) Pam: OK.</Paragraph>
      <Paragraph position="6"> (6) Joe: One, two, three!</Paragraph>
    </Section>
  </Section>
  <Section position="8" start_page="35" end_page="35" type="metho">
    <SectionTitle>
MERGING AUGMENTED RGRAPHS
</SectionTitle>
    <Paragraph position="0"> As discussed thus far, the construction algorithm produces an explanation for how an activity r is related to a goal A. However, to properly model collaboration, one must also take into account the context of previously discussed activities. Thus, we now address how the algorithm explains an activity r in this context.</Paragraph>
    <Paragraph position="1"> Because Gi and Gj are collaborating, it is appropriate for Gj to assume that any activity mentioned by Gi is part of doing A (or at least that Gi believes that it is). If this is not the case, then Gi must explicitly indicate that to Gj (Grosz and Sidner, 1990). Given this assumption, Gj's task is to produce a coherent explanation, based upon her recipes, for how all of the activities that she and Gi discuss are related to A.</Paragraph>
    <Paragraph position="2"> We incorporate this model of Gj's task into the algorithm by requiring that each recipe have at most one corresponding augmented rgraph, and implement this restriction as follows: whenever an rgraph node corresponding to a particular act-type in a recipe is created, the construction algorithm checks to see whether there is Mready another node (in a previously constructed rgraph) corresponding to that act-type. If so, the algorithm tries to merge the augmented rgraph currently under construction with the previous one, in part by merging these two nodes. In so doing, it combines the information contained in the separate explanations.</Paragraph>
    <Paragraph position="3"> The processing of utterance (4) in the sample di-Mogue illustrates this procedure. The activity derived from utterance (4) is r2=(lifl(keyboard(piano)), {pare}, tl). The initial augmented rgraph portion created in explaining this activity is shown in Figure 5. Node N5 of the rgraph corresponds to the act-type simult(lifl(foot(piano)),lift(keyboard(piano))) and includes information derived from r2. But the rgraph (in Figure 4) previously constructed in explaining rl also includes a node, N2, corresponding to this act-type (and containing information derived from rl). Rather than continuing with an independent explanation for r2, the algorithm attempts to combine the information 5The function cover_interval takes a set of time intervals as an argument and returns a time interval spanning the set (Balkanski, 1990).</Paragraph>
    <Paragraph position="4"> from the two activities by merging their augmented rgraphs.</Paragraph>
    <Paragraph position="5">  Two augmented rgraphs are merged by first merging their rgraphs at the two nodes corresponding to the same act-type (e.g. nodes N5 and N2), and then merging their constraints. Two nodes are merged by unifying the activities they represent. If this unification is successful, then the two sets of constraints are merged by taking their union and adding to the resulting set the equality constraints expressing the bindings used in the unification. If this new set of constraints is satisfiable, then the bindings used in the unification are applied to the remainder of the two rgraphs. Otherwise, the algorithm fails: the activities represented in the two rgraphs are not compatible. In this case, because the recipe corresponding to the rgraphs does not provide an explanation for all of the activities discussed by the agents, it is removed from further consideration.</Paragraph>
    <Paragraph position="6"> The augmented rgraph resulting from merging the two augmented rgraphs in Figures 4 and Figure 5 is shown in Figure 6.</Paragraph>
  </Section>
  <Section position="9" start_page="35" end_page="35" type="metho">
    <SectionTitle>
IMPLEMENTATION
</SectionTitle>
    <Paragraph position="0"> An implementation of the algorithm is currently underway using the constraint logic programming language, CLP(7~) (Jaffar and Lassez, 1987; Jaffar and Miehaylov, 1987). Syntactically, this language is very similar to Prolog, except that constraints on real-valued variables may be intermixed with literals in rules and goals. Semantically, CLP(~) is a generalization of Prolog in which unifiability is replaced by solvability of constraints. For example, in Prolog, the predicate X &lt; 3 fails if X is uninstantiated. In CLP(~), however, X &lt; 3 is a constraint, which is solvable if there exists a substitution for X that makes it true.</Paragraph>
    <Paragraph position="1"> Because many of the augmented rgraph constraints are relations over real-valued variables (e.g. the time of one activity must be before the time of another), CLP(T~) is a very appealing language in which to implement the augmented rgraph construction process.</Paragraph>
    <Paragraph position="2"> The algorithm for implementing this process in a logic programming language, however, differs markedly from the intuitive algorithm described in this paper.</Paragraph>
  </Section>
  <Section position="10" start_page="35" end_page="37" type="metho">
    <SectionTitle>
RGRAPHS AND CONSTRAINTS VS. EGRAPHS
</SectionTitle>
    <Paragraph position="0"> Kautz (1987) presented several graph-based algorithms derived from his formal model of plan recognition. In Kautz's algorithms, an explanation for an observation is represented in the form of an explanation graph or egraph.</Paragraph>
    <Paragraph position="1"> Although the term rgraph was chosen to parallel Kautz's terminology, the two representations and algorithms are quite different in scope.</Paragraph>
    <Paragraph position="2"> Two capabilities that an algorithm for plan recognition in collaborative discourse must possess are the abilities to represent joint actions of multiple agents and to reason about hypothetical actions. In addition, such an algorithm may, and for efficiency should, exploit assumptions of the communicative situation. The augmented rgraph representation and algorithm meet these qualifications, whereas the egraph representation and algorithms do not.</Paragraph>
    <Paragraph position="3"> The underlying action representation used in rgraphs is capable of representing complex relations among acts, including simultaneity and sequentiality.</Paragraph>
    <Paragraph position="4"> In addition, relations among the agents and times of acts may also be expressed. The action representation used in egraphs is, like that in STRIPS, simple step decomposition. Though it is possible to represent simultaneous or sequential actions, the egraph representation can only model such actions if they are performed by the same agent. This restriction is in keeping with Kautz's model of keyhole recognition, but is insufficient for modelling intended recognition in multiagent settings.</Paragraph>
    <Paragraph position="5"> Rgraphs are only a part of our representation. Augmented rgraphs also include constraints on the activities represented in the rgraph. Kautz does not have such an extended representation. Although he uses constraints to guide egraph construction, because they are not part of his representation, his algorithm can only check their satisfaction locally. In contrast, by collecting together all of the constraints introduced by the different relations or constructors in a recipe, we can exploit interactions among them to determine unsatisfiability earlier than an algorithm which checks constraints locally. Kautz's algorithm checks each event's constraints independently and hence cannot determine satisfiability until a constraint is ground; it cannot, for example, reason that one constraint makes another unsatisfiable. null Because agents involved in collaboration dedicate a significant portion of their time to discussing the actions they need to perform, an algorithm for rood- null elling plan recognition in discourse must model reasoning about hypothetical and only partially specified activities. Because the augmented rgraph representation allows variables to stand for agents and times in both activities and constraints, it meets this criteria.</Paragraph>
    <Paragraph position="6"> Kautz's algorithm, however, models reasoning about actual event occurrences. Consequently, the egraph representation does not include a means of referring to indefinite specifications.</Paragraph>
    <Paragraph position="7"> In modelling collaboration, unless explicitly indicated otherwise, it is appropriate to assume that all acts are related. In the augmented rgraph construction algorithm, we exploit this by restricting the reasoning done by the algorithm to recipes for A, and by combining explanations for acts as soon as possible. Kautz's algorithm, however, because it is based on a model of keyhole recognition, does not and cannot make use of this assumption. Upon each observation, an independent egraph must be created explaining all possible uses of the observed action. Various hypotheses are then drawn and maintained as to how the action might be related to other observed actions.</Paragraph>
  </Section>
class="xml-element"></Paper>