Realizing Expressions of Doubt in Collaborative Dialogue * 
Leah Schroeder and Sandra Carberry 
Computer and Information Sciences 
University of Delaware 
Newark, DE 19716 
{schroede, carberry}@cis.udel.edu 
Abstract 
One way to 1)egin a negotiation subdialogue is to ex- 
press doubt at a proposition. However, expressions 
of doubt occur in a variety of forms, each of which 
conveys information about the nature of the doubt 
that is important for the subsequent resolution of 
the conflict. This paper presents our work on real- 
izing expressions of doubt appropriately in natural 
 dialogues. 
1 Introduction 
Participants in a collaborative natural  di- 
alogue must develop mutual beliefs about what is 
said, what is meant, and the implications for the task 
at hand. We may think of each utterance as a pro- 
posed change to the agents' common ground (Clark, 
1996). Since autonomous agents enter the dialogue 
with differing domain, world, and personal knowl- 
edge, it is inevitable that some beliefs conveyed by 
an utterance will not be accepted because they con- 
flict with existing beliefs of the agent. However, it 
is also the case that these conflicting belief~ will not 
necessarily result in rejection of the proposed beliefS, 
but in subdialogues to negotiate a modification that 
is acceptable to both agents(Chu-Carroll and Car- 
berry, 1995). One w~y to begin such a subdialoguc 
is to express doubt ~t the beliefs proposed by an 
utterance. In the following example, the boldface 
utterance is expressing doubt at the previous utter- 
ance 1 (Transcripts, 1982)2: 
H: and - there's no reason why you shouldn't havc 
an ira for last yr 
J: well i thought they just started this yr 
* This work was supported by NSF grant ~GER-9354869 
and #CDA-9703088 
1Throughout this paper I use the phrase "doubt at an 
utterance" in place of "doubt at a proposition conveyed or 
implied by an utterance." I do not mean the utterance itself 
is somehow doubted, but that the utterance introduced the 
object of doubt into the dialogue. It may be the case that the 
agent is doubting a proposition expressed in the uttermme, or 
doubting the optimality of, or ability to execute, an action 
suggested in the utterance. 
2All of the examples in this paper, except where otherwise 
noted, are from this source. 
H: oh no. ira's were available as long as you are 
not a participant in an czisting pension 
An e×pression of doubt is an utterance that con- 
veys uncertain disbelief in a proposition that was 
introduced in an earlier utterance. An expression of 
doubt signals that the speaker does not accept the 
utterance at which she is expressing doubt, but she 
is neither expressing a "neutral" attitude toward it 
nor rejecting it with certainty 3. In the above e×am- 
ple, J cammt be said to be rejecting the proposal 
outright, because her response indicates that she is 
uncertain in her disbelief. 
A natural  system must be able to ex- 
press doubt, particularly in cases where it; has in- 
complete or uncertain knowledge. Exmnination of 
natural  corpora shows that expressions of 
doubt may be realized in a variety of forms. Further- 
more, the fbrm of the utterance conveys information 
about the nature of the doubt that is important tbr 
the subsequent resolution of the conflict. Thus a 
collaborative natural  system must be able 
to generate utterances that convey doubt naturally 
and effectively. This paper presents our work on re- 
alizing expressions of doubt appropriately in natural 
 dialogues. 
2 Previous Work 
In Chu-Carroll and Carberry (1998) the collabo- 
rative planning process is modeled as a Propose- 
Evaluate-Modify cycle, in which an agent is able to 
detect conflicts in belief and initiate collaborative 
negotiation subdialogues to attempt to resolve the 
conflicts. They use a modified version of Galliers 
belief revision inechanisln(Galliers, 1992; Logan et 
al., 1994) to determine whether to accept a proposi- 
tion and in determining which conflicting beliefs to 
use to refute an utterance that is not accepted. How- 
ever, their work does not address how an exi)ression 
of doubt should be realized in a natural  
utterance. 
Vander Linden and Di Eugenio (Vander Linden 
and Di Eugenio, 1996) studied negative imperatives 
3Absolute rejection may be expressed as doubt for the sake 
of politeness. We do not address that issue here. 
740 
in instructional texts. They used machine learn- 
lug to correlate features of an action X's relation- 
ship to the reader in terms of attention, awareness, 
and safety, with whether it was realize(t as Don't X, 
Never X, or Take care not to X. In our research, we 
draw on their notion of identifying how features of 
the generation context correlate with how an utter- 
ance should be expressed. However, our work differs 
Dora theirs in that we must deal with an agent's be- 
lie£s motiw~ting his doubt and we consider a wider 
range of variations in realization. 
3 Communicating an Expression of 
Doubt 
We assume appropriate mechmfisms for detecting 
conflict and determining when to engage in a sub- 
dialogue by expressing doul)t(Chu-Carroll and Car- 
berry, 1998), as well as an approi)riate belief revision 
mechmfism, and in this paper concentrate on how an 
expression of doubt should be realized as an utter- 
anee. A cooperative agent should be as informative 
as ne.eded, without expressing too much irreleva.nt 
intbnnation(Grice, 1975). Thus, in formulating an 
expression of doubt, we must consider how much the 
doubted ageut needs to know in order to collaborate 
in resolving the doubt and how much we can ex- 
pect him to infer without being exl)lieitly told. In 
addition, Clark (1.996) argues that particiI)ants in 
discourse select; utterances that express their eom- 
muni(:ative intent efficiently, oft, ca in sentence flag- 
meats. Since such efficiency of ezpression is the ex- 
pected natural form of discourse, a hearer is likely 
to (terive unintended imt)lications from significantly 
less economical realizations. 
Expressions of doubt, by our detinition, signal 
nonacceptance because of 'unccrl, ain disbelief. In 
order for the doubted agent to attemt)t to collab- 
orate in resolving the doubt, he needs to know sev- 
eral things. Most basically, he nee(is to recognize 
that there is doubt in a particular utterance. In 
the absence of an objection to an utterance, the 
speaker will assmne an implicit acceptance(Lmnbert 
and Carberry, 1999). To efficiently negotiate an 
acceptable resolution to the belief conflict, ideally 
the doubted agent must know something about the 
1)ellen of the doubtiug agent; in particular, which 
belief(s) are causing her nonacceptance, and the 
strength of these beliefS. If the doubted agent de- 
cides to retain his original beliet:s, this information 
helps him to construct an argument that will be 
maximally effective and efficient in his attempt to 
convince the doubting agent(Chu-Carroll and Car- 
berry, 1998). 
To i(lentifs~ how expressions of (loubt are realized 
in naturally occurring dialogue and how tiles(; re- 
alizations convey the requisite beliet's, we analyzed 
features of individual ext)ressions of doubt extracted 
from natural corpora, and correlated the various 
forms of the utterances with the features of the un- 
derlying beliefs, t{owever, as explained ill Section 
3.3, the use of machine learning techniques was not 
apt)ropriate due to the nature of our corpus. Sec- 
tion 3.1 discusses features of underlying beliefs and 
Section 3.2 discusses the various forms that an ex- 
pression of doubt can take. Section 3.3 then presents 
it set of rules that relate the two. 
3.1 Belief features 
As noted above, beliefs play a prominent role in 
expressions of doubt, since a speaker will ideally 
convey enough intbrmation tbr the hearer to dis- 
eern 1) that she is expressing doubt, 2) what she is 
doubting, 3) any support she has tbr the doubt, and 
4) the strength of this supi)ort. In addition, speak- 
ers tend to differentiate new SUl)porting int'ornla- 
lion from that which is already part of the COlIlliIOU 
ground aud which should already have been consid- 
ered. These beliefs are often IIot explicitly stated, 
but are assumed to be inferable by the doubted agent 
based on his knowledge of the previous dialogue, 
knowledge of the other agent's belieN, a model of 
stereotypical beliefs, linguistic knowledge, and the 
particular realization of the doubting agent's utter- 
alice. 
For example, consider the following assertion and 
two possible responses, each expressing doubt at; 
the prot)osition P, to~,a that John Smith gets $800 
a month fl'om Social Seeurityd: 
S: ,loh, n Smith, (lets $800 a month in Social Security. 
1) U: Isn't h,e less th, an 62 yem's old? 
2) U: $800'? 
In 1) U relies on illutua,1 donmin knowh:dge to ex- 
press (hml)t at Pdo,,bt by contending some ()tiler 
proposition Pi that implies -,P, to~,bt(Lamllert and 
Carberry, 1999), namely that Slnith is younger than 
62. In the rest of this paper, P, to,,a refers to the 
doubted proposition and Pi to a proposition other 
than Pao,,bt, if any, that is the reason for this doubt. 
In addition, expectations also play a role in ex- 
pressions of doubt. In the simplest case, the violated 
expectation is just that I~to~,bt is fiflse. In other sit- 
uations, an agent may have an expectation that a 
proposition will be false if instmltiated with some 
particular subset of its possible instantiations. Re- 
sponses that conflict with these expectations may 
provoke an expression of doubt, even though the 
doubting agent may have little or no support for 
the ext)ectation. Such violated expectations are of- 
tel1 signaled by elliptical fragments, such as response 
2) above where U conveys not oIfly that she doesn't 
4This is not a naturally occurring example, but was made 
up for exl)lanatory lmrposes. 
741 
accept Pdo~tbt, but also that her doubt steins from 
tile instantiation of tile amount term as 8800. We 
hypothesize that U might accept a t)roposition with 
a different instantiation of tile amount term, lint U 
doesn't explicitly state this, and other instantiations 
may be irrelevant. A violated expectation will be 
referred to as PC and is described further in Sec- 
tion 3.1.2. When and how these expectations arise 
is a topic for fllture research. 
We assume the t)ropositions Pao~,l,t, Pc, and Pi, 
as well as the fact that we want to express doubt, 
as inputs to our generation process. Note that ev- 
ery expression of doubt will be associated with some 
Pdoubt and solne Pc, since for every expression of 
doubt, there must be some doubted proposition and 
some inconsistency between the doubting agent's ex- 
pectations and belief in Pdoubt. There may or may 
not be an associated Pi, depending on the doubt- 
ing agent's beliet~ supporting ~Pdo,,bt. Lack of a Pi 
indicates that tile agent's belief in ~P(to,tbt is unsup- 
ported ~. 
Based on the information that a speaker will ide- 
ally convey when expressing doubt (as discussed at 
the beginning of this section), we hypothesize that 
the following aspects of a speaker's beliefs are sig- 
nificant factors in how an expression of doubt is re- 
alized. 
3.1.1 Features Associated with P,,o,a,t 
Endorsement of Pdo,a,t: Refers to the authority 
behind the asserted proi)osition, which imi)acts the 
strength of tlm hearer's doubt(Chu-Carroll and Car- 
berry, 1998) 
• Expert - The information is coming from a do- 
main expert, or coming from someone with first- 
hand knowledge (including personal preferences). 
• Reliable - The agent suggesting the proposition 
is not an expert, trot is considered a generally knowl- 
edgeable source. 
• Questionable - hffbrmation thai; doesn't come 
fl'om an expert or reliable agent, or that is stated 
uncertainly by such an agent. 
3.1.2 Features Associated with P,, 
Pc. feature: /2~ refers to a violated expectation. In 
the following, we identify three kinds of expectations 
that may be violated by an assertion. For illustra- 
tive purposes, assume that S has made the following 
assertion: 
S: The most you will 9et back on your taxes is $~00. 
• Term-value: 
Pe = False(-I~to,,bt, _term, _value) 
5AIthough human agents may generally be able to offer 
soine weak supt)orl, for their l)ellefs, it is possible, depend- 
ing on the belief revision system used, to have no supporting 
evidence for a belief (Logan et al., 1994). 
Tile doubting agent may fail to accept -P, lo,bt with 
_term instantiated to _value, due to an expectation 
that _value is not one of tile instantiations of _term 
that would make -Pdo,,~t true. For example, the 
hearer of the above assertion by S may have ex- 
pected a much larger vahm than $400, with little 
or no support for this expectation. 
• Constraint: 
P~ = F alse(-P~toubt , _term, _value, _constraint) 
The doubting agent may fail to accept -\]~lo~,bt due 
to an expectation that -Pdo,,~t will be false when 
_term is instantiated with _value, in situations in 
which _constraint holds. This constraint is not a 
term in -Pdo,,~t, lint tile doubting agent believes that 
the speaker of -l~zo,,~t intends that the constraint 
hold. For example, the hearer of the above assertion 
by S may believe that S means $400 for the whole 
3,ear, but may have expected a larger amount unless 
S was referring to, say, quarterly taxes. 
• General: 
P~ = False(-P~lo,bt) 
The doubting agent may fail to accept l%~o,,~t in its 
entirety without having a specific objection to any 
particular term in tile prol)osition. 
3.1.3 Features Associated with Pi ~ 
Commonality of Pi refers to tile source of the 
doubting agent's conflicting belief, if any. 
• Old - A prior conflicting belief is already i)art of 
tile explicit common ground of tile dialogue. 
• New - The doubting agent doesn't believe, that 
her conflicting belief is already part of the common 
ground estat)lished t)y the preceding dialogue. 
Endorsement of Pi refers to the strength of evi- 
dence supporting the belief 1~ that is in eonfiiet with 
the doubted belief. The endorsements are listed here 
from strongest to weakest. 
• First-hand - Belief is a personal t)reference or 
sometlfing directly experienced. 
• Expert - Belief supported 1)y expert testimony, 
or thought to be common knowledge among experts 
in tlfis domain. 
• Reliable - Belief conmmnicated from someone 
who, while not an expert, is generally considered 
a knowledgeable source of information. 
• Default - Belief believed to be common knowl- 
edge, in tim sense that the speaker strongly believes 
it and strongly believes that others who belong to 
a certain community (namely one which she has in 
common with the other dialogue agent) believe it as 
well. 
awe make the simplifying assumption that only one such 
proposition has been identified for use in an expression of 
doubt, as this is 1;he case in all of the expressions of doubt 
we encountered in our corpus. We leave consideration of ex- 
pressing multiple l~'s in one utterance for fllture work. 
742 
• Derived - Belief is (leriv('.d froln other 1)e.liefs in 
such a way that it is considered strong. 
• Ilyl)othesis - The 1)elief is derived fl'om other be- 
liefs in such a way that it; is considered weak. This 
category includes beliefs derived from analogy with 
another belief in a similar 1)roposition. 
• None - The belief is unsupported. 
Endorsement of hnplication ret>rs to the 
strength of evidence SUl)l)Orl;ing the belief that Pi 
being true ilnl)lies thai; Pdo~,bt is not l;rue. The en- 
dorsements are listed from strongest to weakest. Wc 
assume the salne definitions as the category above 
and that the two lists lie on the same strength 
scale. That is, a.n iml)lication endorsed as reliable 
is the same streng(;h as a P+ endorsed as reliable and 
stronger tlmn a I} endorsed as hypothesis. The only 
addition to this list; is the Logical (,Jldorsement to ac- 
(:omd; for instance.s in which P,~,,,,a can I)e logically 
deduced from Pi. 
• l,ogieal - ~P,l<,,,bl dircc(,ly inferred from I}. 
• First-han(l r 
• ExI)ert 
• Reliable 
o l)efault 
• l)erived 
• lIypothesis 
• NOlle 
3.2 Form features 
Expressions of (lollbl; ()c(:ur in a variol;y of forlllS. We 
dis(Anguish l;holn actor(ling to l;h0, surfa(:(~ form o17 l;h(~ 
lll;1Ler~tllce~ tim t)l'eSc, n(:(~ of two clue wor(ls, and ~he 
sl)ecifi('ii;y of tim informal;ion conve, ye(1. 
Surface Form 
• Surface Negative Question - "Isn't that, only 
worth what someone will I)ay ti)r it?" This (;ai;(~gory 
also includes negative tag (tuestions. 
• Simple interrogative- "Can I join the \[RA when 
i am 657" 
• Statement as Question - "I must tile a return?" 
t n s category also in(:ludes ellit)ti(:al fragments su('h 
as "$4007" 
• Siml)le declaration - "I calculated 10." 
• Prot)osition within a belief ('lause - "I thought 
they only started this year." 
Clue word 
• Bill; 
• Even ~hough 
7Tho, (lUO, stion of how l\[lllCh experience is lmedo, d to lO~tl'll 
a Imlief in ;m iml)lical,ion is I)eyond the scope of this 1)a,l)er. 
Specificity - General forms of the expression (;all 
1)e more or less specific in tile amount of intbrnmtion 
COil\Toyed. 
• Generic: Sentence that is a general question of 
the previous utterance. 
h. you .still -you have to file a state income tax 
return as well 
j. i do? 
• Repetition: Rel)etition of a phrase from i)revious 
utterance. 
h. OK, what I'd likc you to do is lend h, im the 20 
thousand. 
1. 20 thousand? 
• l/ei)etition+ : l~,et)etitioil of t)hrase froln 1)revious 
utterance t)lus new intbrmation 
h. rigM,, the mazimum amount of credit that you 
will b(', able to get will be $/~00 on their taz rot'am 
e. $400 for the whole year? 
• Contradict;: Presentation of a 1)citer t, hat iinl)li(!s 
the negation of \])do,,bt 
h. and th, crc's no reason why you shouldn't have 
an ira for last yr 
j. well i thought they just started this yr 
• Contra(licl;+Source: Pre, senta.tion of a (:ontra(lic- 
l,()ry lmlieJ' and th(' sour(',e of that 1)elief. 
h had told j he nlust t)ay tax on his $6256 
j. ram. h, arry another th, in.q. i have the internal 
uh revenue uh ask you about that 6256 
$ uh since i have the fund he said no! ,so 
wh, at do i do now? 
• Explicit+Contradict: ExI)licit statement of dis- 
belief, followed by a contradictory belief. 
b. well ah, h,c uh, ... h,c belongs to a money mar- 
kct )t'nd now and uh, th, ey will do that \[invest 
it in govt securities as part of thcir individual 
retirement accou'nt\] for him 
h. i'm not so sure of it.. they may move it 
ah in(;() a into a govt securities fund, 1)ut 
i'm not so sure that they can move it into 
individual securities - check that out 
3.3 Realizing an Expression of Doubt 
Many of the exi)ressions of doubt in our COlI)tlS are 
non-ideal, t)ecause they were not recognized as doul)t 
or because information that was not included in 
t, he utterance, lint could have been, was ultimately 
needed to resolve the doul)t. Thus it was not al)l)ro- 
priate to use the corlms as training data tbr machine 
learning. Consequently, tile following rules are l)ased 
743 
and implication beliefs that would have caused tile 
form of expression of doubt to be generated. We also 
encouraged subjects to write in beliet's which were 
not inchlded, but none (lid. Out of the 60 instances 
(ten expressions of doubt times six subjects), tim 
subjects five times chose beliefs that we did not rep- 
resent as contributing to the doubt and three times 
failed to recognize a belief that (lid contribute. 
The subjects also rated the beliefs according to 
strength. We evaluated these ratings to see if the 
communicated strengths were correlated with the 
endorsements of beliefs that would have generated 
this form. Since subjects varied in the ranges that 
I;hey used in rating the strengths of the beliefs, we 
looked at tile scores relative to each subjects ratings 
of the other beliefs. 
Most of the strength ratings were consistent with 
the rules. The most fi'equent inconsistency was the 
case in which we would have generated a form based 
on slightly different endorsements for Pi and the in> 
plication, but our subjects rated them equivalent 
strengths. While it may be the case that tmotfle 
don't actually perceive a difli;renee, it may also be 
the case that numerical ratings don't fully capture 
the same information that t, he notion of endorse- 
ments do. 
The only significant inconsistency with our rules 
was one utterance ill which doubt was expressed by 
"1 thought that, but my husl)and, lie wasn't sure if 
that just uleant ss pension." We had represented the, 
husband as a relial/le source an(l t;hus generated ill- 
formatioll about the, source of the conflicting propo 
sit;ion. In this instance, the doul)t was not judged 
very strong l)y our subje(:ts, and tlm agent's t)eli(~f 
in her husband's exl)ertise as relatively weak. In 
future work, we will further explore exi)ression:s of 
doubt for which it is imi)ortant to (:omnmnicate the 
som'(:e of inforlnation. 
\Ve consider this a l)reliminary (;valuation to show 
that the rules we have l'ormulated thus fitr are re;> 
sortable, l?urther evahlation will tie neexle, d to pro- 
vide cvitlem:e that subjects really do draw &J.\[('rcnt 
inferences based on the different forms of exl)ression 
and that our rules accurately captnr(. ~ these ditti?r- 
eUCeS. 
5 Conclusion and Future work 
This pape.r has 1)resented rules that could be used 
by a natural  system to realize exl)ressions 
of doubt. We have identified sew;ral forms that are 
used to express doubt ill naturally occurring dia- 
logues. Our rules correlate these forms to 1)eliet~ 
of the doul)ting agent, takiug into consideration the 
l)eliefs that must be conveyed tbr the utterance to 
be a successflll exl)ression of doubt. 
Preliininary evaluation shows that the belief fea- 
ture values in our rules correspond to hmnan sul> 
jects' intuitions about tile strength of tile doubt. In 
addition, the beliefs that would generate each t:onn 
are consistent with the belief~ that the subjects at- 
trilmte to tile doubting agent when that form was 
used. 
Future work will e(meentrate on refining the fea- 
tures and exploring more explicit reasoning about 
tile beliefs of the other agent. We also plan to ex- 
plore the role of intonation in realizing expressions 
of doubt. 

References 

Jennifer Chu-Carroll and Sandra Carberry. 1995. 
Comnmnieation for conflict resolution iu multi- 
agent collaborative planning. In Proceedings of 
the .Inter'national CoT@rcncc on Multi-Agent Sys- 
tems, pages 49 56. 

Jennifer Chu-.Carroll and Saudra Carberry. 1998. 
Collaborative response generation in planning di- 
alogues. Computational Linguistics, 24(3):355 
400. 

Iterbert Clark. 1996. Using Langua9('. Cambridge 
University Press. 

Julia. Rose Galliers. 1992. Autonomous belief revi- 
sion and communic.ation, in P. Gardenf'or.% esli- 
tor, Belief Revision, Cambridge tracts ill theoreti- 
cal colnlmter science. Cambridg(; University Press, 
Cambridge, England. 

H. Paul Grice. 1975. Logic and Conversation. In 
P. Cole and J. L. Morgan, editors, Synta:c and ,%- 
ma'u, tic.s IIl: Speech Acts, pages 4t 58, N.Y. Aca- 
demic Press. 

Lynn Lambert and Sandra Carberry. 1999. A pro- 
tess model for recognizing communicative acts 
and modeling negol, iation subdialogues. Co'mpu- 
tatio'nal \]Ang'aistics , 25:1 54. 

Brian Logan, St, even II.c, ece, Allison Cawsey, Julia 
Galliers, and Karen Sparck .lones. :1994. Belief 
revision and dialogue management in int'ormat;iou 
retrieval. T(.'chnicaI report, University of (Jan> 
bridge Computer lmboratory. 

ra , ra ~q * Harry Gross lranscrlpts. 1982. \[ran, crq)ts derived 
Kern tapes of the radio talk show harry gross: 
Streaking of your money. Provided by the l)epl;. 
of Comlmter Science at the Universii,y of Pemlsyi-- 
vania. 

Keith Vmlder Linden and Barbara l)i Eugenio. 1996. 
A corpus study of negative imlleratives in natm'ai 
 instruci;ionso In Proceedings of the 15th. 
International Confl'.rcnce on Uomputat, io'n, al Lin- 
gusitics (COLINU-96)~ Copenhagen. 
