<?xml version="1.0" standalone="yes"?>
<Paper uid="P94-1001">
  <Title>I I User I I NL Input I NL Interpretation Modules Observed Conversation Acts Dialogue \] Manager j~ I Domain Directives I Domain Task Interaction 1 Modules '~'1 I I NL Output j- I NL Generation 1 Module Intended Conversation Acts Domain Observations and Directive Responses</Title>
  <Section position="5" start_page="0" end_page="2" type="metho">
    <SectionTitle>
3 Obligations and Discourse Obligations
</SectionTitle>
    <Paragraph position="0"> Obligations represent what an agent should do, according to some set of norms. The notion of obligation has been studied for many centuries, and its formal aspects are examined using Deontic Logic. Our needs are fairly simple, and do not require an extensive survey of the complexities that arise in that literature. Still, the intuitions underlying that work will help to clarify what an obligation is. Generally, obligation is defined in terms of a modal operator often called permissible.</Paragraph>
    <Paragraph position="1"> An action is obligatory if it is not permissible not to do it.</Paragraph>
    <Paragraph position="2"> An action is forbidden if it is not permissible. An informal semantics of the operator can be given by positing a set of rules of behavior R. An action is obligatory if its occurrence logically follows from R, and forbidden if its non-occurrence logically follows from R. An action that might occur or notoccur according to R is neither obligatory nor forbidden.</Paragraph>
    <Paragraph position="3"> Just because an action is obligatory with respect to a set of rules R does not mean that the agent will perform the action.</Paragraph>
    <Paragraph position="4"> So we do not adopt the model suggested by \[Shoham and Tennenholtz, 1992\] in which agents' behavior cannot violate the defined social laws. If an obligation is not satisfied, then this means that one of the rules must have been broken.</Paragraph>
    <Paragraph position="5"> We assume that agents generally plan their actions to violate as few rules as possible, and so obligated actions will usually occur. But when they directly conflict with the agent's personal goals, the agent may choose to violate them. Obligations are quite different from and can not be reduced to intentions and goals. In particular, an agent may be obliged to do an action that is contrary to his goals (for example, consider a child who has to apologize for hitting her younger brother).</Paragraph>
    <Paragraph position="6"> Obligations also cannot be reduced to simple expectations, although obligations may act as a source of expectations.</Paragraph>
    <Paragraph position="7"> Expectations can be used to guide the action interpretation and plan-recognition processes (as proposed by \[Carberry, 1990\]), but expectations do not in and of themselves provide a sufficient motivation for an agent to perform the expected action - in many cases there is nothing wrong with doing the unexpected or not performing an expected action. The interpretation of an utterance will often be clear even without coherence with prior expectations. We need to allow for the possibility that an agent has performed an action even when this violates expectations. If an agent actually violates obligations as well then the agent can be held accountable. 1 Specific obligations arise from a variety of sources. In a conversational setting, an accepted offer or a promise will incur an obligation. Also, a command or request by the other party will bring about an obligation to perform the requested action. If the obligation is to say something then we call this a discourse obligation. Our model of obligation is very simple. We use a set of rules that encode discourse conventions. Whenever a new conversation act is determined 1 \[McRoy, 1993\] uses expectations derived from Adjacency Pair structure \[Schegloff and Sacks, 1973\], as are many of the discourse obligations considered in this paper. These expectations correspond to social norms and do impose the same notion of accountability. However, the analysis there is oriented towards discovering misconceptions based on violated expectations, and the alternative possibility of violated obligations is not considered in the utterance recognition process, nor allowed in the utterance production process.</Paragraph>
    <Paragraph position="8">  to have been performed, then any future action that can be inferred from the conventional rules becomes an obligation.</Paragraph>
    <Paragraph position="9"> We use a simple forward chaining technique to introduce obligations.</Paragraph>
    <Paragraph position="10"> Some obligation rules based on the performance of conversation acts are summarized in Table 1. When an agent performs a promise to perform an action, or performs an acceptance of a suggestion or request by another agent to perform an action, the agent obliges itself to achieve the action in question. When another agent requests that some action be performed, the request itself brings an obligation to address the request: that is either accept it or to reject it (and make the decision known to the requester) - the requestee is not permitted to ignore the request. A question establishes an obligation to answer the question. If an utterance has not been understood, or is believed to be deficient in some way, this brings about an obligation to repair the utterance.</Paragraph>
    <Paragraph position="11"> source of obligation obliged action</Paragraph>
    <Section position="1" start_page="2" end_page="2" type="sub_section">
      <SectionTitle>
3.1 Obligations and Behavior
</SectionTitle>
      <Paragraph position="0"> Obligations (or at least beliefs that the agent has obligations) will thus form an important part of the reasoning process of a deliberative agent, e.g., the architecture proposed by \[Bratman et al., 1988\]. In addition to considering beliefs about the world, which will govern the possibility of performing actions and likelyhood of success, and desires or goals which will govern the utility or desirability of actions, a social agent will also have to consider obligations, which govern the permissibility of actions.</Paragraph>
      <Paragraph position="1"> There are a large number of strategies that may be used to incorporate obligations into the deliberative process, based on how much weight they are given compared to the agents goals. \[Conte and Castelfranchi, 1993\] present several strategies of moving from obligations to actions, including: automatically performing an obligated action, adopting all obligations as goals, or adopting an obligated action as a goal only when performing the action results in a state desired by the agent. In the latter cases, these goals still might conflict with other goals of the agent, and so are not guaranteed to be performed.</Paragraph>
      <Paragraph position="2"> In general, we will want to allow action based on obligations to supersede performance of intended actions. For instance, consider an agent with an intention to do something as soon as possible. If an obligation is imposed, it will still be possible to perform the intended action, but a well-behaved agent might need to delay performance until the obligation is dealt with. For example, if the intention is to perform a series of inform acts, and then a listener requests repair of one, a well-behaved agent will repair that inform before proceeding to initiate the next intended one.</Paragraph>
    </Section>
  </Section>
  <Section position="6" start_page="2" end_page="19" type="metho">
    <SectionTitle>
4 Using Discourse Obligations in a Dialogue
</SectionTitle>
    <Paragraph position="0"/>
    <Section position="1" start_page="2" end_page="19" type="sub_section">
      <SectionTitle>
System
</SectionTitle>
      <Paragraph position="0"> We have built a system that explicitly uses discourse obligations and communicative intentions to partake in natural dialogue. This system plays the role of the dialogue manager in the TRAINS dialogue system which acts as an intelligent planning assistant in a transportation domain. While this is a domain where the assumption of co-operation is generally valid, the obligation model still provides for a much simpler analysis of the discourse behavior than a strongly plan-based account. An example of a dialogue that the TRAINS system can engage in is shown in Figure 1. Below we describe parts of the discourse model in more detail and then show how it is used to account for aspects of this dialogue.</Paragraph>
      <Paragraph position="1">  Speaker: Utterance U: We better ship a boxcar of oranges to Bath by 8 AM.</Paragraph>
      <Paragraph position="2"> S : Okay.</Paragraph>
      <Paragraph position="3"> U: So we need to get a boxcar to Coming where there are oranges.</Paragraph>
      <Paragraph position="4">  U: There are oranges at Corning.</Paragraph>
      <Paragraph position="5"> U: Right? S : Right.</Paragraph>
      <Paragraph position="6"> U: So we need an engine to move the boxcar.</Paragraph>
      <Paragraph position="7"> U: Right? S : Right.</Paragraph>
      <Paragraph position="8"> U: So there's an engine at Avon.</Paragraph>
      <Paragraph position="9"> U: Right? S : Right.</Paragraph>
      <Paragraph position="10"> U: So we should move the engine at Avon, engine El, to Dansville to pick up the boxcar there.</Paragraph>
      <Paragraph position="11"> S : Okay.</Paragraph>
      <Paragraph position="12"> U: And move it from Dansville to Corning. U: Load up some oranges into the boxcar. U: And then move it on to Bath.</Paragraph>
      <Paragraph position="13">  The TRAINS System \[Allen and Schubert, 1991\] is alarge integrated natural language conversation and plan reasoning 2This is a slightly simplified version of a spoken dialogue between two people. The original is dialogue 91-6.1 from \[Gross et al., 1993\]. The utterance numbering system used here reflects the relation to the turn and utterance numbering used there. '3-7' represents utterance 7 within turn 3. '=' is used to indicate merged utterances. Thus '3-3=6' spans four utterances in turn 3 of the original, and 9=13 replaces turns 9 through 13 in the original.  system. We concentrate here, however, on just one part of that system, the discourse actor which drives the actions of the dialogue manager module. Figure 2 illustrates the system from the viewpoint of the dialogue manager.</Paragraph>
      <Paragraph position="14">  chitecture of the TRAINS Conversation System The dialogue manager is responsible for maintaining the flow of conversation and making sure that the conversational goals are met. For this system, the main goals are that an executable plan which meets the user's goals is constructed and agreed upon by both the system and the user and then that the plan is executed.</Paragraph>
      <Paragraph position="15"> The dialogue manager must keep track of the current state of the dialogue, determine the effects of observed conversation acts, generate utterances back, and send commands to the domain plan reasoner and domain plan executor when appropriate. Conversational action is represented using the theory of Conversation Acts \[Traum and Hinkelman, 1992\] which augments traditional Core Speech Acts with levels of acts for turn-taking, grounding \[Clark and Schaefer, 1989\], and argumentation. Each utterance will generally contain acts (or partial acts) at each of these levels.</Paragraph>
    </Section>
    <Section position="2" start_page="19" end_page="19" type="sub_section">
      <SectionTitle>
4.1 Representing Mental Attitudes
</SectionTitle>
      <Paragraph position="0"> As well as representing general obligations within the temporal logic used to represent general knowledge, the system also maintains two stacks (one for each conversant) of pending discourse obligations. Each obligation on the stack is represented as an obligation type paired with a content. The stack structure is appropriate because, in general, one must respond to the most recently imposed obligation first. As explained in Section 4.2, the system will attend to obligations before considering other parts of the discourse context. Most obligations will result in the formation of intentions to communicate something back to the user. When the intentions are formed, the obligations are removed from the stack, although they have not yet actually been met. If, for some reason, the system dropped the intention without satisfying it and the obligation were still current, the system would place them back on the stack.</Paragraph>
      <Paragraph position="1"> The over-riding goal for the TRAINS domain is to construct and execute a plan that is shared between the two participants. This leads to other goals such as accepting proposals that the other agent has suggested, performing domain plan synthesis, proposing plans to the other agent which the domain plan reasoner has constructed, or executing a completed plan.</Paragraph>
    </Section>
    <Section position="3" start_page="19" end_page="19" type="sub_section">
      <SectionTitle>
4.2 The Discourse Actor Algorithm
</SectionTitle>
      <Paragraph position="0"> In designing an agent to control the behavior of the dialogue manager, we choose a reactive approach in which the system will not deliberate and add new intentions until after it has performed the actions which are already intended. As shown above, though, new obligations will need to be addressed before performing intended actions. The agent's deliberative behavior could thus be characterized in an abstract sense as: loop perceive world and update beliefs if system has obligations then address obligations else if system has performable intentions then perform actions else deliberate on goals When deciding what to do next, the agent first considers obligations and decides how to update the intentional structure (add new goals or intentions) based on these obligations. Obligations might also lead directly to immediate action. If there are no obligations, then the agent will consider its intentions and perform any actions which it can to satisfy these intentions. If there are no performable intentions, then the system will deliberate on its overall goals and perhaps adopt some new intentions (which can then be performed on the next iteration).</Paragraph>
      <Paragraph position="1"> For the discourse actor, special consideration must be given to the extra constraints which participation in a conversation imposes. This includes some weak general obligations (such as acknowledging utterances by others and not interrupting) as well as some extra goals coming from the domain setting to maintain a shared view of the world and the domain plans which are to be executed. We prioritize the sources for the deliberations of the actor as follows:  1. Discourse Obligations from Table 1 2. Weak Obligation: Don't interrupt user's turn 3. Intended Speech Acts 4. Weak Obl: Grounding (coordinate mutual beliefs) 5. Discourse Goals: Domain Plan Negotiation 6. High-level Discourse Goals  The implemented actor serializes consideration of these sources into the algorithm in Figure 3.</Paragraph>
      <Paragraph position="2">  (1) while conversation is not finished (2) if system has obligations (3) then address obligations (4) else if system has turn (5) then if system has intended conversation acts (6) then call generator to produce NL utterances (7) else if some material is ungrounded (8) then address grounding situation (9) else if some proposal is not accepted (I 0) then consider proposals (11 ) else if high-level goals are unsatisfied (12) then address goals (13) else release turn or attempt to end conversation (I 4) else if no one has turn (15) then take turn (16) else if long pause (17) then take turn  The updating of the conversational state due to perceived conversation acts or actions of other modules of the system progresses asynchronously with the operation of the discourse actor. Whenever the discourse actor is active, it will first decide on which task to attempt, according to the priorities given in Figure 3, and then work on that task. After completing a particular task, it will then run through the loop again, searching for the next task, although by then the context may have changed due to, e.g., the observance of a new utterance from the user. The actor is always running and decides at each iteration whether to speak or not (according to turn-taking conventions); the system does not need to wait until a user utterance is observed to invoke the actor, and need not respond to user utterances in an utterance by utterance fashion.</Paragraph>
      <Paragraph position="3"> Lines 2-3 of the algorithm in Figure 3 indicate that the actor's first priority is fulfilling obligations. If there are any, then the actor will do what it thinks best to meet those obligations. If there is an obligation to address a request, the actor will evaluate whether the request is reasonable, and if so, accept it, otherwise reject it, or, if it does not have sufficient information to decide, attempt to clarify the parameters. In any case, part of meeting the obligation will be to form an intention to tell the user of the decision (e.g., the acceptance, rejection, or clarification). When this intention is acted upon and the utterance produced, the obligation will be discharged. Other obligation types are to repair an uninterpretable utterance or one in which the presuppositions are violated, or to answer a question. In question answering, the actor will query its beliefs and will answer depending on the result, which might be that the system does not know the answer.</Paragraph>
      <Paragraph position="4"> In most cases, the actor will merely form the intention to produce the appropriate utterance, waiting for a chance, according to turn-taking conventions, to actually generate the utterance. In certain cases, though, such as a repair, the system will actually try to take control of the turn and produce an utterance immediately. For motivations other than obligations, the system adopts a fairly &amp;quot;relaxed&amp;quot; conversational style; it does not try to take the turn until given it by the user unless the user pauses long enough that the conversation starts to lag (lines 14-17). When the system does not have the turn, the conversational state will still be updated, but the actor will not try to deliberate or act.</Paragraph>
      <Paragraph position="5"> When the system does have the turn, the actor first (after checking obligations) examines its intended conversation acts. If there are any, it calls the generator to produce an utterance 3 (lines 5-6 of the discourse actor algorithm). Whatever utterances are produced are then reinterpreted (as indicated in Figure 2) and the conversational state updated accordingly. This might, of course, end up in releasing the turn. It might not be convenient to generate all the intended acts in one utterance, in which case there will remain some intended acts left for future utterances to take care of (unless the subsequent situation merits dropping those intentions). Only intended speech acts that are part of the same argumentation acts as those which are uttered will be kept as intentions - others will revert back to whatever caused the intention to be formed, although subsequent deliberation might cause the intentions to be re-adopted.</Paragraph>
      <Paragraph position="6"> If there are no intended conversation acts, the next thing the actor considers is the grounding situation (lines 7-8). The actor will try to make it mutually believed (or grounded) whether particular speech acts have been performed. This will involve acknowledging or repairing user utterances, as well as repairing and requesting acknowledgement of the system's own utterances. Generally, groundingis considered less urgent than acting based on communicative intentions, although some grounding acts will be performed on the basis of obligations which arise while interpreting prior utterances. If all accessible utterances are grounded, the actor then considers the negotiation of domain beliefs and intentions (lines 9-10). The actor will try to work towards a shared domain plan, adding intentions to perform the appropriate speech acts to work towards this goal. This includes accepting, rejecting, or requesting retraction of user proposals, requesting acceptance of or retracting system proposals, and initiating new system proposals or counterproposals.</Paragraph>
      <Paragraph position="7"> The actor will first look for User proposals which are not shared. If any of these are found, it will add an intention to accept the proposal, unless the proposal is deficient in some way (e.g., it will not help towards the goal or the system has already come up with a better alternative). In this latter case, the system will reject the user's proposal and present or argue for its own proposal. Next, the actor will look to see if any of its own proposals have not been accepted, requesting the user to accept them if they have been, or retracting or reformulating them if they have already been rejected. Finally, the actor will check its private plans for 3Actually, if the only utterance is an acknowledgement, the actor will postpone the production until it checks that there is nothing else that it can combine in the same utterance, such as an acceptance or answer.</Paragraph>
      <Paragraph position="8">  any parts of the plan which have not yet been proposed. If it finds any here, it will adopt an intention to make a suggestion to the user.</Paragraph>
      <Paragraph position="9"> If none of the more local conversational structure constraints described above require attention, then the actor will concern itself with its actual high-level goals. For the TRAINS system, this will include making calls to the domain plan reasoner and domain executor, which will often return material to update the system's private view of the plan and initiate its own new proposals. It is also at this point that the actor will take control of the conversation, pursuing its own objectives rather than responding to those of the user.</Paragraph>
      <Paragraph position="10"> Finally, if the system has no unmet goals that it can work towards achieving (line 13), it will hand the turn back to the user or try to end the conversation if it believes the user's goals have been met as well.</Paragraph>
    </Section>
    <Section position="4" start_page="19" end_page="19" type="sub_section">
      <SectionTitle>
4.3 Examples
</SectionTitle>
      <Paragraph position="0"> The functioning of the actor can be illustrated by its behavior in the dialogue in Figure 1. While the discussion here is informal and skips some details, the dialogue is actually processed in this manner by the implemented system. More detail both on the dialogue manager and its operation on this example can be found in \[Traum, 1994\].</Paragraph>
      <Paragraph position="1"> Utterance 1 is interpreted both (literally) as the initiation 4 of an inform about an obligation to perform a domain action (shipping the oranges). This utterance is also seen as an (indirect) suggestion that this action be the goal of a shared domain plan to achieve the performance of the action. In addition, this utterance releases the turn to the system. Figure 4 shows the relevant parts of the discourse state after interpretation of this utterance.</Paragraph>
      <Paragraph position="2">  After interpreting utterance 1, the system first decides to acknowledge this utterance (lines 7-8 in the actor algorithm) - moving the suggestion from an unacknowledged to unaccepted - and then to accept the proposal (lines 9-10). Finally, the system acts on the intentions produced by these deliberations (lines 4-5) and produces the combined acknowledgement/acceptance of utterance 2. This acceptance makes the goal shared and also satisfies the first of the discourse goals, that of getting the domain goal to work on.</Paragraph>
      <Paragraph position="3"> Utterances 3-3=6 and 3-7 are interpreted, but not responded to yet since the user keeps the turn (in this case by following up with subsequent utterances before the system has a chance to act). Utterance 3-8 invokes a discourse 4According to the theory of Conversation Acts \[Traum and Hinkelman, 1992\], Core Speech Acts such as inform are multi-agent actions which have as their effect a mutual belief, and are not completed unless/until they are grounded.</Paragraph>
      <Paragraph position="4"> obligation on the system to respond to the User's assertion in 3-7 and also gives the turn to the system. The resulting discourse context (after the system decides to acknowledge)  The system queries its domain knowledge base and decides that the user is correct here (there are, indeed, oranges at Coming), and so decides to meet this obligation (lines 2-3) by answering in the affirmative. This results in forming an intention to inform, which is then realized (along with the acknowledgement of the utterances) by the generation of utterance 4.</Paragraph>
      <Paragraph position="5"> Similar considerations hold for the system responses 6 and 8. The reasoning leading up to utterance 14 is similar to that leading to utterance 2. Here the user is suggesting domain actions to help lead to the goal, and the system, when it gets the turn, acknowledges and accepts this suggestion.</Paragraph>
      <Paragraph position="6"> Utterances 15-2--4, 15-5=7, and 15-8=10 are interpreted as requests because of the imperative surface structure. The discourse obligation to address the request is incurred only when the system decides to acknowledge the utterances and ground them. After the decision to acknowledge, the obligations are incurred, and the system then addresses the requests, deciding to accept them all, and adding intentions to perform an accept speech act, which is then produced as 16.</Paragraph>
      <Paragraph position="7"> Utterance 17 is interpreted as a request for evaluation of the plan. When the system decided to acknowledge, this creates a discourse obligation to address the request. The system considers this (invoking the domain plan reasoner to search the plan for problems or incomplete parts) and decides that the plan will work, and so decides to perform the requested action - an evaluation speech act. This is then generated as 18-3. The discourse state after the decision to acknowledge  After the user's assent, the system then checks its goals, and, having already come up with a suitable plan, executes this plan in the domain by sending the completed plan to the domain plan executor.</Paragraph>
      <Paragraph position="8"> This example illustrates only a small fraction of the capabilities of the dialogue model. In this dialogue, the system needed only to follow the initiative of the user. However this architecture can handle varying degrees of initiative, while remaining responsive. The default behavior is to allow the user to maintain the initiative through the plan construction phase of the dialogue. If the user stops and asks for help, or even just gives up the initiative rather than continuing with further suggestions, the system will switch from plan recognition to plan elaboration, and will incrementally devise a plan to satisfy the goal (although this plan would probably not be quite the same as the plan constructed in this dialogue). We can illustrate the system behaving more on the basis of goals than obligations with a modification of the previous example. Here, the user releases the turn back to the system after utterance 2, and the deliberation proceeds as follows: the system has no obligations, no communicative intentions, nothing is ungrounded, and there are no unaccepted proposals, so the system starts on its high-level goals. Given its goal to form a shared plan, and the fact that the current plan (consisting of the single abstract move-commodS_ty action) is not executable, the actor will call the domain plan reasoner to elaborate the plan. This will return a list of augmentations to the plan which can be safely assumed (including a move- eng 5_ne event which generates the movecommodity, given the conditions that the oranges are in a boxcar which is attached to the engine), as well as some choice point where one of several possibilities could be added (e.g., a choice of the particular engine or boxcar to use).</Paragraph>
      <Paragraph position="9"> Assuming that the user still has not taken the turn back, the system can now propose these new items to the user. The choice could be resolved in any of several ways: the domain executor could be queried for a preference based on prior experience, or the system could put the matter up to the user in the form of an alternative question, or it could make an arbitrary choice and just suggest one to the user.</Paragraph>
      <Paragraph position="10"> The user will now be expected to acknowledge and react to these proposals. If the system does not get an acknowledgement, it will request acknowledgement the next time it considers the grounding situation. If the proposal is not accepted or rejected, the system can request an acceptance. If a proposal is rejected, the system can negotiate and offer a counterproposal or accept a counter proposal from the user.</Paragraph>
      <Paragraph position="11"> Since the domain plan reasoner \[Ferguson, 1994\] performs both plan recognition and plan elaboration in an incremental fashion, proposals from system and user can be integrated naturally in a mixed-initiative fashion. The termination condition will be a shared executable plan which achieves the goal, and each next action in the collaborative planning process will be based on local considerations.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>