<?xml version="1.0" standalone="yes"?>
<Paper uid="J99-1001">
  <Title>A Process Model for Recognizing Communicative Acts and Modeling Negotiation Subdialogues</Title>
  <Section position="8" start_page="41" end_page="45" type="evalu">
    <SectionTitle>
6. Evaluation and Future Work
</SectionTitle>
    <Paragraph position="0"> We undertook an evaluation of our prototype system both to assess whether it derived appropriate interpretations of utterances and to identify areas for further research. We obtained eight human volunteers, six of whom are not engaged in NLP research and two of whom are involved in unrelated NLP projects. The subjects were given a set of world knowledge stereotypically believed in the domain, such as that faculty on sabbatical do not normally teach. The subjects were presented with a set of dialogues and asked to analyze several utterances from each dialogue. The selected utterances did not include simple questions initiating the dialogue or straightforward answers to questions, since it seemed likely that the subjects would agree with the system's interpretation and thus the results would be biased in favor of the system. The selected utterances did include surface negative questions (both with and without a clue word but), statements interpreted by our system as support for a previous assertion or as explanations about why a proposition was not in conflict with a previous claim, and examples of implicit acceptance.</Paragraph>
    <Paragraph position="1"> For each utterance selected for analysis, the subjects were given a suggested interpretation, and asked whether the suggested interpretation was reasonable and whether they could identify a better interpretation. 27 For 15 of 20 utterances, the subjects unanimously believed that the system's interpretation was best. It should be noted there was unanimous agreement that utterance (42) below should be interpreted as an expression of doubt but that utterance (39) should not.</Paragraph>
    <Paragraph position="2">  EA: Who is teaching architecture? CA: Dr. Smith is teaching architecture.</Paragraph>
    <Paragraph position="3"> EA: Isn't Dr. Smith an excellent teacher? Dialogue B (40) EA: Who is teaching architecture? (41) CA: Dr. Smith is teaching architecture.</Paragraph>
    <Paragraph position="4"> (42) EA: But isn't Dr. Smith an excellent teacher?  There were two categories of utterances where the subjects disagreed. In the case of surface negative questions that did not express doubt, such as utterance (39) above, 27 The subjects were not told that the suggested interpretation was the one produced by our system but only that we were trying to determine how utterances in a discourse should be interpreted.  Carberry and Lambert Modeling Negotiation Subdialogues the suggested interpretation given to the subjects was that the speaker was seeking information about whether the queried proposition was true. When the subjects did not interpret the utterance as an expression of doubt (see below), five of them contended that a better interpretation would be that EA was seeking verification of the queried proposition. Since our system already recognizes from the surface negative question that the speaker has a strong (but uncertain) belief in the queried proposition, it is easy to extend our system so that it can explicitly identify a Seek-Veri~cation discourse act.</Paragraph>
    <Paragraph position="5"> The other category for which there was disagreement was surface negative questions where a clue word was not present and the stereotypical domain knowledge did not provide a conflict. In two of five instances, some subjects used their own experience to identify a mutual belief that might suggest a conflict, such as the belief that sometimes certain faculty are not allowed to teach graduate level courses. While this knowledge cannot be captured as a default rule, it does represent a kind of shared experiential knowledge that would provide weak evidence for a potential conflict. However, it should be noted that our subjects were split on how these problematic cases should be interpreted, agreeing with the system's interpretation slightly more than half the time. There was also another such surface negative question where one subject viewed the system's interpretation as reasonable but argued that an expression of doubt would be a better interpretation. In order to derive this interpretation, the subject posited an attribute for the speaker that was neither evident from the dialogue nor stereotypically true. (The other subjects agreed that the system's interpretation was best.) These examples bear on the issue of accommodation mentioned in Section 4.5.1, since one could argue that the subjects who interpreted the utterances as expressions of doubt were trying to accommodate an incompatibility. This is particularly true in the last instance where the subject found it necessary to resort to nonshared knowledge in making the interpretation. However, it is unclear whether a speaker would expect a listener to recognize such utterances as expressions of doubt without additional clues. As noted below, our future research will consider other forms of evidence (gestural and intonational) in order to resolve such ambiguous utterances.</Paragraph>
    <Paragraph position="6"> After they had finished analyzing the dialogues, we asked the subjects to construct three dialogues containing an expression of doubt and to explain why the expression of doubt should be interpreted as such. While these dialogues provided no contradictions to our approach, they did provide a couple of interesting examples, such as the following dialogue, that suggest areas for future work.</Paragraph>
    <Paragraph position="7">  (43) EA: We have basil, parsley, and oregano, but we need marjoram.</Paragraph>
    <Paragraph position="8"> (44) CA: Isn't marjoram the same as oregano?  Clearly (44) is expressing doubt at the claim conveyed by (43), but it relies on shared world knowledge that if a list contains X items, the X items are presumed to be different. Our system does not currently include such knowledge.</Paragraph>
    <Paragraph position="9"> Our subjects commented that intonation and facial gesture might alter their interpretation of the utterances in the dialogues; we are beginning research that will take these kinds of evidence into account (Carberry, Chu-Carroll, and Lambert 1996). In addition, we will be expanding the kinds of world knowledge incorporated into our system, and will be considering both the strength of different pieces of evidence and how several pieces of weak evidence affect interpretation. We would also like to extend our use of linguistic clues to include a wide variety of clue words and phrases and to recognize the functions that these words can play. In addition, we are developing a plan-based response generation component (Chu-Carroll and Carberry 1994).  Computational Linguistics Volume 25, Number 1 Initial work on this component includes a subsystem that can identify what evidence to present to a user when conflicts arise (Chu-Carroll and Carberry 1995b, 1998) and what information to request when the system cannot rationally decide whether to accept a proposition conveyed by the user (Chu-Carroll and Carberry 1995a, 1998). We will also be investigating the scale-up of our system as we extend its coverage. Part of the motivation for the content of the current discourse recipes was their future extension to other domains, such as tutoring. For example, as discussed in Section 4.2.1, the formulation of our Ask-Ref recipe allows it to be used as a subaction of a future Test-Knowledge discourse act since the recipe does not presume that the speaker is ignorant about the correct value of the requested term. This should aid in extending the kinds of discourse acts that can be handled. Although transporting our system to another domain will require encoding new domain knowledge and new domain recipes, the recipes for discourse and problem-solving acts are domain-independent and thus will remain unchanged. Moreover, the knowledge captured in our recipes is communicative knowledge shared by dialogue participants; we believe that such communicative knowledge (such as how to express doubt) is finite although the possible intentions (such as the intention of expressing doubt at Dr. Smith teaching CS360) are infinite.</Paragraph>
    <Paragraph position="10">  7. Other Related Work</Paragraph>
    <Section position="1" start_page="43" end_page="43" type="sub_section">
      <SectionTitle>
7.1 Grosz and Sidner's Theory of Discourse Processing
</SectionTitle>
      <Paragraph position="0"> Grosz and Sidner (1986) postulated a theory of discourse structure that included linguistic, intentional, and attentional components, and they argued that the dominance and satisfaction-precedes relationships between discourse segments must be identified in order to determine discourse structure. They also noted three kinds of information that contribute to determining the purposes of discourse segments and their relationship to one another: linguistic markers, utterance-level intentions, and general knowledge about actions and objects. Subsequently Lochbaum (1994) developed an algorithm based on Grosz and Sidner's SharedPlan model (Grosz and Sidner 1990) that recognizes discourse segment purposes and discourse structure.</Paragraph>
      <Paragraph position="1"> We contend that, in order to understand utterances and respond appropriately, it is necessary not only to determine the structure of the discourse but also to identify the communicative acts that an agent intends to perform with an utterance. 2s For example, if a listener does not recognize when an utterance such as &amp;quot;Wasn't Dr. Smith on campus yesterday?&amp;quot; is expressing doubt, then the listener's response might fail to address the reasons for this doubt. Our research provides a computational algorithm that uses multiple knowledge sources to recognize complex discourse acts, including expressions of doubt, and to identify their relationship to one another. This algorithm and our strategy for recognizing implicit acceptance enable us to model negotiation subdialogues, something that previous systems have been unable to handle.</Paragraph>
    </Section>
    <Section position="2" start_page="43" end_page="44" type="sub_section">
      <SectionTitle>
7.2 Argument Understanding Systems
</SectionTitle>
      <Paragraph position="0"> Several researchers have built argument understanding systems, but none has addressed participants coming to an agreement or mutual belief about a particular situation, either because the researchers investigated monologues only (Cohen 1987; Cohen and Young 1991), or because they assumed that dialogue participants do not change 28 In a dialogue, Grosz and Sidner's discourse segment purpose is intended to capture the purpose of a segment consisting of a series of utterances by both participants, not the communicative intentions underlying each participant's discourse actions.</Paragraph>
      <Paragraph position="1">  Carberry and Lambert Modeling Negotiation Subdialogues their minds (Flowers, McGuire, and Birnbaum 1982; Quilici 1991). Cohen (1987) developed an argument understanding system that used clue words and an evidence oracle to build a discourse structure for arguments based on which utterances served as support for other utterances. Cohen's model, however, handles only monologues, so responses to arguments are not modeled in her system. Birnbaum, Flowers, Dyer, and McGuire (Flowers and Dyer 1984; McGuire, Birnbaum, and Flowers 1981; Birnbaum, Flowers, and McGuire 1980) developed a system that finds flaws in arguments and determines how to respond. Quilici (1991) created a system in which agents respond to each other's arguments based on a justification pattern that will support the agent's position. Both Quilici and Birnbaum et al., however, assume that all participants in an argument will retain their opinion throughout the course of the argument, and concentrate mainly on how to find flaws in arguments and construct responses based on those findings; they do not address actually winning arguments. Reichman (1981) modeled informal debates by using her idea of context spaces and expectations to determine who should respond and what possible topics might be addressed. However, she does not provide a detailed computational mechanism for recognizing the role of each utterance in a debate.</Paragraph>
    </Section>
    <Section position="3" start_page="44" end_page="45" type="sub_section">
      <SectionTitle>
7.3 Models of Collaborative Behavior
</SectionTitle>
      <Paragraph position="0"> Several models of discourse have recently been built which view conversation as a kind of collaborative behavior in which speakers try to make themselves understood and listeners work with speakers to help speakers attain this goal.</Paragraph>
      <Paragraph position="1"> Clark and Schaefer (1989) contend that utterances must be &amp;quot;grounded,&amp;quot; or understood, by both parties, but they do not address conflicts in belief, only lack of understanding. Walker (1992) has found many occasions of redundancy in collaborative dialogues, and explains these by claiming that people repeat themselves in order to ensure that each utterance has been understood. 29 Clark and Wilkes-Gibbs (1990) propose a collaborative model of dialogue in which referring is viewed as a collaborative process and each conversation unit is viewed as a contribution, which consists of 1) an utterance that performs a referring action, and 2) the utterances required to understand the referent described in the utterance. Heeman (1991) implemented this model in a plan-based collaborative model of dialogue that is able to plan and recognize referring expressions and their corrections.</Paragraph>
      <Paragraph position="2"> Other collaborative models assume that two participants are working together to achieve a common goal (Cohen and Levesque 1990a, 1991a, 1991b; Lochbaum, Grosz, and Sidner 1990; Lochbaum 1991; Grosz and Sidner 1990; Searle 1990). Searle (1990) proposes a model in which the two agents working together have a joint intention, a &amp;quot;we intention,&amp;quot; instead of individual intentions. Cohen and Levesque (1990a, 1990b, 1990c, 1991a, 1991b) have developed a formal theory in which agents are jointly committed to accomplishing a goal, so both parties have individual intentions to accomplish the goal as part of their joint commitment. Grosz, Lochbaum, and Sidner (Grosz and Sidner 1990; Lochbaum, Grosz, and Sidner 1990; Lochbaum 1991) have specified a system in which two agents are working to accomplish some common goal by building a &amp;quot;shared plan&amp;quot; in which each agent holds certain beliefs and intentions. These beliefs and intentions indicate that the agents intend to perform some joint action, and that they believe they can perform this action. All of these models indicate the need for modeling collaborative dialogue, but none suggests a system that can handle the 29 Another reason for repetition, she claims, is for centering (Grosz, Joshi, and Weinstein 1995), but she concentrates on repetitions that give evidence of understanding.</Paragraph>
      <Paragraph position="3">  Computational Linguistics Volume 25, Number 1 kind of negotiation subdialogues that people often engage in when trying to negotiate their conflicts in belief, even when they are both working towards the same goal.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>