Proceedings of the 7th SIGdial Workshop on Discourse and Dialogue, pages 28–35,
Sydney, July 2006. c©2006 Association for Computational Linguistics
Interactive Question Answering and Constraint Relaxation
in Spoken Dialogue Systems
Sebastian Varges
CSLI
Stanford University
Stanford, CA 94305, USA
varges@stanford.edu
Fuliang Weng, Heather Pon-Barry
Research and Technology Center
Robert Bosch Corporation
4009 Miranda Ave, Palo Alto, CA, USA
fuliang.weng, heather.pon-barry
@rtc.bosch.com
Abstract
We explore the relationship between ques-
tion answering and constraint relaxation in
spoken dialog systems. We develop dia-
logue strategies for selecting and present-
ing information succinctly. In particular,
we describe methods for dealing with the
results of database queries in information-
seeking dialogs. Our goal is to structure
the dialogue in such a way that the user is
neither overwhelmed with information nor
left uncertain as to how to refine the query
further. We present evaluation results ob-
tained from a user study involving 20 sub-
jects in a restaurant selection task.
1 Introduction
Information presentation is an important issue
when designing a dialogue system. This is espe-
cially true when the dialogue system is used in a
high-stress environment, such as driving a vehi-
cle, where the user is already occupied with the
driving task. In this paper, we explore efficient
dialogue strategies to address these issues, and
present implemented knowledge management, di-
alogue and generation components that allow cog-
nitively overloaded users – see (Weng et al., 2004),
for example – to obtain information from the di-
alogue system in a natural way. We describe a
knowledge manager that provides factual and on-
tological information, a content optimizer that reg-
ulates the amount of information, and a genera-
tor that realizes the selected content. The domain
data is divided between domain-specific ontolo-
gies and a database back-end. We use the system
for both restaurant selection and MP3 player tasks,
and conducted experiments with 20 subjects.
There has been substantial previous work on
information presentation in spoken dialogue sys-
tems. (Qu and Green, 2002) also present a
constraint-based approach to cooperative informa-
tion dialogue. Their experiments focus on over-
constrained queries, whereas we also deal with un-
derconstrained ones. Moreover, we guide the user
through the dialogue by making suggestions about
query refinements, which serve a similar rˆole to
the conditional responses of (Kruijff-Korbayova et
al., 2002). (Hardy et al., 2004) describe a dialogue
system that uses an error-correcting database man-
ager for matching caller-provided information to
database entries. This allows the system to se-
lect the most likely database entry, but, in contrast
to our approach, does not modify constraints at
a more abstract level. In contrast to all the ap-
proaches mentioned above, our language gener-
ator uses overgeneration and ranking techniques
(Langkilde, 2000; Varges and Mellish, 2001).
This facilitates variation and alignment with the
user utterance.
A long-standing strand of research in NLP is
in natural language access to databases (Androut-
sopoulos et al., 1995). It mainly focused on map-
ping natural language input to database queries.
Our work can be seen as an extension of this work
by embedding it into a dialogue system and al-
lowing the user to refine and relax queries, and
to engage in clarification dialogs. More recently,
work on question answering (QA) is moving to-
ward interactive question answering that gives the
user a greater role in the QA process (HLT, forth-
coming). QA systems mostly operate on free text
whereas we use a relational database. (Thus, one
needs to ‘normalize’ the information contained in
free text to use our implemented system without
further adaption.)
28
In the following section, we give an overview
of the dialogue system. We then describe the
knowledge management, dialogue and generation
components in separate sections. In section 6 we
present evaluation results obtained from a user
study. This is followed by a discussion section and
conclusions.
2 System architecture
Our dialogue system employs the following archi-
tecture: the output of a speech recognizer (Nu-
ance, using a statistical language model) is ana-
lyzed by both a general-purpose statistical depen-
dency parser and a (domain-specific) topic classi-
fier. Parse trees and topic labels are matched by
the ‘dialogue move scripts’ of the dialogue man-
ager (Mirkovic and Cavedon, 2005; Weng et al.,
2005). The scripts serve to license the instantia-
tion of dialogue moves and their integration into
the ‘dialogue move tree.’ The use of dialogue
move scripts is motivated by the need to quickly
tailor the system to new domains: only the scripts
need to be adapted, not the underlying machinery
implemented in Java. The scripts define short se-
quences of dialog moves, for example a command
move (“play song X”) may be followed either by
a disambiguation question or a confirmation that
the command will be executed. A dialogue pro-
ceeds by integrating such scripted sequences into
the dialogue move tree, yielding a relatively ‘flat’
dialogue structure.
Query constraints are built by dialogue move
scripts if the parse tree matches input patterns
specified in the scripts. These query constraints
are the starting point for the processing strategies
described in this paper. The dialogue system is
fully implemented and has been used in restau-
rant selection and MP3 player tasks. There are 41
task-independent, generic dialogue move scripts,
52 restaurant selection scripts and 89 MP3 player
scripts. The examples in this paper are mostly
taken from the restaurant selection task.
3 Knowledge and Content management
The Knowledge Manager (KM) controls access to
domain knowledge that is structured according to
domain-dependent ontologies. The KM makes use
of OWL, a W3C standard, to represent the onto-
logical relationships between domain entities. The
knowledge base can be dynamically updated with
new instances at any point. In a typical interac-
tion, the Dialog Manager converts a user’s query
into a semantic frame (i.e., a set of semantic con-
straints) and sends this to the KM via the content
optimizer. For example, in the Restaurant domain,
a request such as “I want to find an inexpensive
Japanese restaurant that takes reservations” results
in the semantic frame below, where Category is a
system property, and the other constraints are in-
herited properties of the Restaurant class:
(1) system:Category = restaurant:Restaurant
restaurant:PriceLevel = 0-10
restaurant:Cuisine = restaurant:japanese
restaurant:Reservations = yes
In addition to the KM module, we employ a
Content Optimization (CO) module that acts as
an intermediary between dialogue and knowledge
management during the query process. It receives
semantic frames from the Dialogue Manager, re-
vises the semantic frames if necessary (see below),
and queries the Knowledge Manager.
The content optimizer also resolves remaining
ambiguities in the interpretation of constraints.
For example, if the user requests an unknown cui-
sine type, the otherwise often accurate classifier
will not be able to provide a label since it oper-
ates under a closed-world assumption. In contrast,
the general purpose parser may be able to pro-
vide an accurate syntactic analysis. However, the
parse still needs to be interpreted by the content
optimizer which has the domain-specific knowl-
edge to determine that “Montenegrin restaurant”
is a cuisine constraint rather than a service level
constraint, for example. (See also section 7).
Depending on the items in the query result set,
configurable properties, and (potentially) a user
model, the CO module selects and performs an ap-
propriate optimization strategy. To increase porta-
bility, the module contains a library of domain-
independent strategies and makes use of external
configuration files to tailor it to specific domains.
The CO module can modify constraints de-
pending on the number of items in the result
set, the system ontology, and information from
a user model. Constraints can be relaxed, tight-
ened, added or removed. The manner in which
a constraint is modified depends on what kind of
values it takes. For example, for the Cuisine
constraint, values are related hierarchically (e.g.,
Chinese, Vietnamese, and Japanese are all sub-
types of Asian), whereas PriceLevel values are
linear (e.g., cheap, moderate, expensive), and
acceptsCreditCards values are binary (e.g., ac-
29
cepted or not accepted).
If the original query returns no results, the con-
tent optimizer selects a constraint to modify and
then attempts to relax the constraint value. If re-
laxation is impossible, it removes the constraint
instead. Constraint relaxation makes use of the
ontological relationships in the knowledge base.
For example, relaxing a Cuisine constraint entails
replacing it with its parent-concept in the domain
ontology. Relaxing a linear constraint entails re-
placing the current value with an adjacent value.
Relaxing a binary constraint entails replacing the
current value with its opposite value.
Based on the ontological structures, the content
optimizer also calculates statistics for every set of
items returned by the knowledge manager in re-
sponse to a user’s query. If the result set is large,
these figures can be used by the dialogue manager
to give meaningful responses (e.g., in the MP3 do-
main, “There are 85 songs. Do you want to list
them by a genre such as Rock, Pop, or Soul?”).
The content optimizer also produces constraints
that represent meta-knowledge about the ontology,
for example, in response to a user input “What
cuisines are there?”:
(2) rdfs:subClassOf = restaurant:Cuisine
The processing modules described in the next
sections can use meta-level constraints in similar
ways to object-level constraints (see (1)).
4 Dialogue strategies for dealing with
query results
In the following two sections, we describe how
our dialogue and generation strategies tie in with
the choices made by the content optimizer. Con-
sider the following discourse-initial interaction for
which the semantic frame (1) is constructed:
(3)
U: i want to find an inexpensive Japanese
restaurant that takes reservations
S: I found 9 inexpensive Japanese
restaurants that take reservations .
Here are the first few :
S: GINZA JAPANESE RESTAURANT
S: OKI SUSHI CAFE
S: YONA SUSHI
S: Should I continue?
The example query has a relatively small result
set which can be listed directly. This is not always
the case, and thus we need dialogue strategies that
deal with different result set sizes. For example, it
does not seem sensible to produce “I found 2500
restaurants. Here are the first few: ...”. At what
point does it become unhelpful to list items? We
do not have a final answer to this question – how-
ever, it is instructive that the (human) wizard in
our data collection experiments did not start list-
ing when the result set was larger than about 10
items. In the implemented system, we define di-
alogue strategies that are activated at adjustable
thresholds.
Even if the result set is large and the system
does not list any result items, the user may still
want to see some example items returned for the
query. This observation is based on comments
by subjects in experimental dry-runs that in some
cases it was difficult to obtain any query result at
all. For example, speech recognition errors may
make it difficult to build up a sufficiently complex
query. In response to this, we always give some
example items even if the result set is large. (An
alternative would be to start listing items after a
certain number of dialogue turns.) Furthermore,
the system should encourage the user to refine the
query by suggesting constraints that have not been
used yet. This is done by maintaining a list of con-
straints in the generator that is used up as the di-
alogue progresses. This list is roughly ordered by
how likely the constraint will be useful. For exam-
ple, using cuisine type is suggested before propos-
ing to ask for information about reservations or
credit cards.
In our architecture, information flows from the
CO module to the generator (see section 5) via the
dialogue move scripts of the dialogue manager.
These are conditioned on the size of the final re-
sult set and whether or not any modifications were
performed. Table 1 summarizes the main dialogue
strategies. These dialogue strategies represent im-
plicit confirmations and are used if NLU has a high
confidence in its analysis of the user utterance (see
(Varges and Purver, 2006) for more details on our
handling of robustness issues). Small result sets
up to a threshold t1 are listed in a single sentence.
For medium-sized result sets up to a threshold t2,
the system starts listing immediately. For large re-
sult sets, the generator shows example items and
makes suggestions as to what constraint the user
may use next. If the CO module performs any con-
straint modification, the first, constraint realizing
sentence of the system turns reflects the modifica-
tion. (‘NP-original’ and ‘NP-optimized’ in table 1
are used for brevity and are explained in the next
section.)
30
|resultfinal| mod example realization fexp
s1a 0 no I’m sorry but I found no restaurants on Mayfield Road that serve Mediterranean food. 0
s1b 0 yes I’m sorry but I found no [NP-original]. I did not even find any [NP-optimized]. 0
s2a small: no There are 2 cheap Thai restaurants in Lincoln in my database: Thai Mee Choke and 61
> 0,< t1 Noodle House.
s2b small yes I found no cheap Greek restaurants that have a formal dress code but there are 0
4 inexpensive restaurants that serve other Mediterranean food and have a formal
dress code in my database: ... .
s3a medium: no I found 9 restaurants with a two star rating and a formal dress code that are open 212
>= t1,< t2 for dinner and serve French food. Here are the first ones: ... .
s3b medium yes I found no [NP-original]. However, there are N [NP-optimized]. Here are the first few: ... . 5
s4a large: no I found 258 restaurants on Page Mill Road, for example Maya Restaurant , 300
>= t2 Green Frog and Pho Hoa Restaurant. Would you like to try searching by cuisine?
s4b large yes I found no [NP-original]. However, there are N [NP-optimized]. Would you like to try 16
searching by [Constraint]?
Table 1: Dialogue strategies for dealing with query results (last column explained in sec. 6)
5 Generation
The generator produces turns that verbalize the
constraints used in the database query. This is
important since the system may miss or misinter-
pret constraints, leading to uncertainty for the user
about what constraints were used. For this rea-
son, a generic system response such as “I found 9
items.” is not sufficient.
The input to the generator consists of the name
of the dialogue move and the relevant instantiated
nodes of the dialogue move tree. From the in-
stantiated move nodes, the generator obtains the
database query result including information about
query modifications. The core of the generator is
a set of productions1 written in the Java Expert
System Shell (Friedman-Hill, 2003). We follow
the bottom-up generation approach for production
systems described in (Varges, 2005) and perform
mild overgeneration of candidate moves, followed
by ranking. The highest-ranked candidate is se-
lected for output.
Productions map individual database con-
straints to phrases such as “open for lunch”,
“within 3 miles” and “a formal dress code”, and
recursively combine them into NPs. This includes
the use of coordination to produce “restaurants
with a 5-star rating and a formal dress code”,
for example. The NPs are integrated into sen-
tence templates, several of which can be combined
to form an output candidate turn. For example,
a constraint realizing template “I found no [NP-
1Productions are ‘if-then’ rules that operate over a shared
knowledge base of facts.
original] but there are [NUM] [NP-optimized] in
my database” (see below for further explanation)
can be combined with a follow-up sentence tem-
plate such as “You could try to look for [NP-
constraint-suggestion]”.
The selection of which sentence template to use
is determined by the dialogue move scripts. Typ-
ically, a move-realizing production produces sev-
eral alternative sentences. On the other hand, the
NP generation rules realize constraints regardless
of the specific dialogue move at hand. This al-
lows us to also use them for clarification ques-
tions based on constraints constructed from classi-
fier information if the parser and associated parse-
matching patterns fail; all that is required is a new
sentence template, for example “Are you looking
for [NP]?”. We currently use 102 productions
overall in the restaurant and MP3 domains, 38 of
them to generate NPs that realize 19 possible input
constraints (for both domains).
The decision of the CO module to relax or re-
move constraints also affects the generator: there
are two sets of constraints, an ‘original’ one di-
rectly constructed from the user utterance, and an
‘optimized’ one used by the KM module to obtain
the query result (see section 3). In case of con-
straint modifications, these two sets are not identi-
cal but often overlapping. To avoid generating sep-
arate sets of NPs independently for the two con-
straint sets, we assign unique indices to the con-
straints and hand the generator two index sets as
targets of NP generation. We overgenerate NPs
and check their index sets before integrating them
into sentence templates.
31
turn speaker utterance
1.1 User i need a restaurant with a five star food rating excellent service
1.2 System I found 425 restaurants with a five star rating and excellent service , for example
EL CAPITAN FISH MARKET , K-NINE COUNTRY CLUB , and KAISERHOF GERMAN
AMERICAN . You could try to look for cuisine type .
1.3 User which restaurants take reservations
1.4 System I found 353 restaurants that have a five star rating , have excellent service and
take reservations , for example GRAND CENTRAL CAFE , CHASIN CHICKEN , and
CHON WOO RESTAURANT . Would you like to try searching by price ?
Figure 1: Example dialogue 1 from system evaluation using restaurant selection task
Ranking of candidate output moves is done by
using a combination of factors. First, the ranker
computes an alignment score for each candidate,
based on its ngram-based overlap with the user
utterance. For example, this allows us to prefer
“restaurants that serve Chinese food” over “Chi-
nese restaurants” if the user used a wording more
similar to the first. We note that the Gricean
Maxim of Brevity, applied to NLG in (Dale and
Reiter, 1995), suggests a preference for the sec-
ond, shorter realization. However, if the user
thought it necessary to use “serves”, maybe to
avoid confusion of constraints or even to correct an
earlier mislabeling, then the system should make
it clear that it understood the user correctly by
using those same words, thus preferring the first
realization. Mild overgeneration combined with
alignment also allows us to map the constraint
PriceLevel=0-10 in example (1) above to both
“cheap” and “inexpensive”, and use alignment to
‘play back’ the original word choice to the user.
As these examples show, using alignment for rank-
ing in NLG allows one to employ overgeneration
techniques even in situations where no corpus data
is available.2
Second, ranking uses a variation score to ‘cycle’
over sentence-level paraphrases. In the extreme
case of repeated identical user inputs, the system
simply chooses one paraphrase after the other, and
starts over when all paraphrases have been used.
Third, we use an ngram filter based on bad
examples ngrams, removing, for example, “Chi-
nese cheap restaurants” but keeping “cheap Chi-
nese restaurant.” For generalization, we replace
constraint realizations with semantic tags derived
from the constraint names (except for the head
noun), for example the trigram ‘CUISINE PRICE
restaurants’. An alternative is to use a more com-
2However, we do have wizard-of-oz data to inform the
system design (see section 7).
plex grammar formalism to prevent ungrammati-
cal candidate moves.
6 Evaluation
We conducted experimental studies involving 20
subjects in a MP3 player task and 20 subjects in a
restaurant selection task. In the following, we con-
centrate on the restaurant selection task because it
is more challenging for constraint handling and in-
formation presentation.
Each subject in the restaurant selection task
was given 9 scenario descriptions involving 3 con-
straints. Subjects were instructed to use their own
words to find a fitting restaurant. We use a back-
end database of 2500 restaurants containing the
following information for each restaurant: restau-
rant name, cuisine type, city and street names,
service level, rating, whether they accept credit
cards or reservations, price level, open hours, dress
code, additional information (e.g. vegetarian food)
and distance from current location. Some of these
constraints have a fixed number of values (e.g.
service level and dress code), whereas those tak-
ing named entities as values are obviously open-
ended.
Figures 1 and 2 show two of the dialogues from
the experiments. To exemplify the challenges
the system is faced with, user turns are shown
in the form of the output of the speech recog-
nizer (we currently use the best ASR hypothesis
as input to the parser). The task of the first dia-
logue (figure 1) was to find a restaurant that has
the following constraints: [Reservations=yes, Ser-
viceLevel:excellent, Rating=5star]. Despite some
minor speech recognition errors, the system cor-
rectly builds a query for the two constraints pro-
vided by the user in (1.1.) and verbalizes these in
(1.2) to show its understanding of the user input.
It also gives some example items and makes a sug-
gestion. This suggestion is not taken up by the user
32
turn speaker utterance
2.1 User breakfast
2.2 System Are you looking for a restaurant that is open for breakfast ?
2.3 User yes located on bower street
2.4 System I found 71 restaurants on bower street that serve breakfast , for example ,
SOUPLANTATION-TO-GO , RAMAYANI WESTWOOD , and , PHILLY WEST .
You could try to look for cuisine type .
2.5 User which
2.6 System I’m sorry, I do not understand what you mean by: which
2.7 System Please try rephrasing that.
2.8 User casual dress
2.9 System There are 51 restaurants on bower street that serve breakfast and have a casual
dress code in my database , for example , BERTY’S , LABOTETEGA , and ,
JAVAN RESTAURANT . Would you like to try searching by price ?
2.10 User british
2.11 System I found no restaurants on bower street that serve british food , have a casual
dress code and are open for breakfast but there are 8 restaurants on bower street that are open for
breakfast , serve European food and have a casual dress code in my database . Here are the first few :
2.12 System HOTEL BEL-AIR
Figure 2: Example dialogue 2 from system evaluation using restaurant selection task
– in fact, we find that suggestions are generally not
taken up by the user. We believe this is due to the
nature of the tasks, which specified exactly which
criteria to match. On the other hand, in more open
application scenarios, where users may not know
what questions can be asked, suggestions may be
useful. In (1.3) the user issues a sub-query that
further constrains the result set. By again summa-
rizing the constraints used, the system confirms in
(1.4) that it has interpreted the new constraint as a
revision of the previous query. The alternative is
to start a new query, which would be wrong in this
context.
The task of the second dialogue, figure 2, was to
find a restaurant that meets the constraints [Busi-
nessHours:breakfast, StreetName=’bower street’,
DressCode=casual]. This user tends to give
shorter, keyword-style input to the system (2.1,
2.8). In (2.3), the user reacts to a clarification
question and adds another constraint which the
system summarizes in (2.4). (2.5) is an ASR er-
ror which the system cannot handle (2.6, 2.7). The
user constraint of (2.8) is correctly used to revise
the query (2.9), but “british” (2.10) is another ASR
error that leads to a cuisine constraint not intended
in the scenario/by the user. This additional con-
straint yields an empty result set, from which the
system recovers automatically by relaxing the hi-
erarchically organized cuisine constraint to “Eu-
ropean food”. In (2.11) the system uses dialogue
strategy s3b for medium-sized result sets with con-
straint modifications (section 4). The result of
both dialogues is that all task constraints are met.
We conducted 20 experiments in the restaurant
domain, 2 of which were restarted in the middle.
Overall, 180 tasks were performed involving 1144
user turns and 1818 system turns. Two factors con-
tributing to the higher number of system turns are
a) some system turns are counted as two turns,
such as 2.6, 2.7 in figure 2, and b) restaurants in
longer enumerations of result items are counted as
individual turns. On average, user utterances are
significantly shorter than system utterances (4.9
words, standard deviation σ = 3.82 vs 15,4 words,
σ = 13.53). This is a result of the ‘constraint sum-
maries’ produced by the generator. The high stan-
dard deviation of the system utterances can be ex-
plained by the above-mentioned listing of individ-
ual result items (e.g. utterance (2.12) in figure 2).
We collected usage frequencies for the dia-
logue strategies presented in section 4: there was
no occurrence of empty final result sets (strat-
egy s1a/b) because the system successfully re-
laxed constraints if it initially obtained no results.
Strategy s2a (small result sets without modifica-
tions) was used for 61 inputs, i.e. constraint sets
constructed from user utterances. Strategy s3a/b
(medium-sized result sets) was used for 217 times
and required constraint relaxations in 5 cases.
Strategy s4a/b (large result sets) was used for
33
316 inputs and required constraint relaxations in
16 cases. Thus, the system performed constraint
modifications in 21 cases overall. All of these
yielded non-empty final result sets. For 573 in-
puts, no modification was required. There were no
empty final result set despite modifications.
On average, the generator produced 16 output
candidates for inputs of two constraints, 160 can-
didates for typical inputs of 3 constraints and 320
candidates for 4 constraints. Such numbers can
easily be handled by simply enumerating candi-
dates and selecting the ‘best’ one.
Task completion in the experiments was high:
the subjects met all target constraints in 170 out of
180 tasks, i.e. completion rate was 94.44%. An
error analysis revealed that the reasons for only
partially meeting the task constraints were varied.
For example, in one case a rating constraint (“five
stars”) was interpreted as a service constraint by
the system, which led to an empty result set. The
system recovered from this error by means of con-
straint relaxation but the user seems to have been
left with the impression that there are no restau-
rants of the desired kind with a five star rating.
7 Discussion
Based on wizard-of-oz data, the system alter-
nates specific and unspecific refinement sugges-
tions (“You could search by cuisines type” vs “Can
you refine your query?”). Furthermore, many of
the phrases used by the generator are taken from
wizard-of-oz data too. In other words, the sys-
tem, including the generator, is informed by em-
pirical data but does not use this data directly (Re-
iter and Dale, 2000). This is in contrast to genera-
tion systems such as the ones described in (Langk-
ilde, 2000) and (Varges and Mellish, 2001).
Considering the fact that the domain ontology
and database schema are known in advance, it is
tempting to make a closed world assumption in
the generator (which could also help system de-
velopment and testing). However, this seems too
restrictive: assume, for example, that the user has
asked for Montenegrin food, which is an unknown
cuisine type, and that the statistical parser com-
bined with the parse-matching patterns in the di-
alogue manager has labeled this correctly. The
content optimization module will remove this con-
straint since there is no Montenegrin restaurant in
the database. If we now want to generate “I did not
find any restaurants that serve Montenegrin food
...”, we do need to be able to use generation input
that uses unseen attribute-value pairs. The price
one has to pay for this increased robustness and
flexibility is, of course, potentially bad output if
NLU mislabels input words. More precisely, we
find that if any one of the interpretation modules
makes an open-world assumption, the generator
has to do as well, at least as long as we want to
verbalize the output of that module.
7.1 Future work
Our next application domain will be in-car naviga-
tion dialogues. This will involve dialogues that de-
fine target destinations and additional route plan-
ning constraints. It will allow us to explore the
effects of cognitive constraints due to changing
driving situations on dialogue behavior. The nav-
igation domain may also affect the point of inter-
action between dialogue system and external de-
vices: we may query a database to disambiguate
proper names such as street names as soon as these
are mentioned by the user, but start route planning
only when all planning constraints are collected.
An option for addressing the current lack of a
user model is to extend the work in (Cheng et al.,
2004). They select the level of detail to be com-
municated to the user by representing the driver’s
route knowledge to avoid repeating known infor-
mation.
Another avenue of future research is to automat-
ically learn constraint relaxation strategies from
(appropriately annotated) evaluation data. User
modeling could be used to influence the order in
which refinement suggestions are given and deter-
mine the thresholds for the information presenta-
tion moves described in section 4.
One could handle much larger numbers of gen-
eration candidates either by using packing (Langk-
ilde, 2000) or by interleaving rule-based genera-
tion with corpus-based pruning (Varges and Mel-
lish, 2001) if complexity should become an issue
when doing overgeneration.
8 Conclusions
We described strategies for selecting and present-
ing succinct information in spoken dialogue sys-
tems. Verbalizing the constraints used in a query is
crucial for robustness and usability – in fact, it can
be regarded as a special case of providing feed-
back to the user about what the system has heard
and understood (see (Traum, 1994), for example).
34
The specific strategies we use include ‘backing-
off’ to more general constraints (by the system)
or suggesting query refinements (to be requested
explicitly by the user). Our architecture is config-
urable and open: it can be parametrized by em-
pirically derived values and extended by new con-
straint handling techniques and dialogue strate-
gies. Constraint relaxation techniques have widely
been used before, of course, for example in syn-
tactic and semantic processing. The presented pa-
per details how these techniques, when used at the
content determination level, tie in with dialogue
and generation strategies. Although we focussed
on the restaurant selection task, our approach is
generic and can be applied across domains, pro-
vided that the dialogue centers around accessing
and selecting potentially large amounts of factual
information.
Acknowledgments This work is supported by
the US government’s NIST Advanced Technology
Program. Collaborating partners are CSLI, Robert
Bosch Corporation, VW America, and SRI Inter-
national. We thank the many people involved in
system design, development and evaluation, and
the reviewers of this paper.

References
Ion Androutsopoulos, G.D. Ritchie, and P. Thanisch.
1995. Natural Language Interfaces to Databases –
An Introduction. Natural Language Engineering,
1(1):29–81.
Hua Cheng, Lawrence Cavedon, and Robert Dale.
2004. Generating Navigation Information Based on
the Driver’s Route Knowledge. In Proceedings of
the Coling 2004 Workshop on Robust and Adap-
tive Information Processing for Mobile Speech In-
terfaces, pages 31–38, Geneva, Switzerland.
Robert Dale and Ehud Reiter. 1995. Computational
Interpretations of the Gricean Maxims in the Gener-
ation of Referring Expressions. Cognitive Science,
19:233–263.
Ernest Friedman-Hill. 2003. Jess in Action: Java
Rule-Based Systems. Manning Publications.
Hilda Hardy, Tomek Strzalkowski, Min Wu, Cristian
Ursu, Nick Webb, Alan Biermann, R. Bryce Inouye,
and Ashley McKenzie. 2004. Data-driven strategies
for an automated dialogue system. In Proceedings
of the 42nd Meeting of the Association for Compu-
tational Linguistics (ACL’04), Main Volume, pages
71–78, Barcelona, Spain, July.
forthcoming. Proceedings of the workshop on Interac-
tive Question Answering at HLT-NAACL 2006.
Ivana Kruijff-Korbayova, Elena Karagjosova, and Stef-
fan Larsson. 2002. Enhancing collaboration with
conditional responses in information-seeking dia-
logues. In Proc. of 6th workshop on the semantics
and pragmatics of dialogue (EDILOG-02).
Irene Langkilde. 2000. Forest-based Statistical Sen-
tence Generation. In Proc NAACL-00, pages 170–
177.
Danilo Mirkovic and Lawrence Cavedon. 2005. Prac-
tical Plug-and-Play Dialogue Management. In Pro-
ceedings of the 6th Meeting of the Pacific Associa-
tion for Computational Linguistics (PACLING).
Yan Qu and Nancy Green. 2002. A Constraint-based
Approach for Cooperative Information-Seeking Di-
alogue. In Proceedings of the International Work-
shop on Natural Language Generation (INLG-02).
Ehud Reiter and Robert Dale. 2000. Building Applied
Natural Language Generation Systems. Cambridge
University Press, Cambridge, UK.
David Traum. 1994. A Computational Theory
of Grounding in Natural Language Conversation.
Ph.D. thesis, Computer Science Dept., U. Rochester.
Sebastian Varges and Chris Mellish. 2001. Instance-
based Natural Language Generation. In Proc.
NAACL-01.
Sebastian Varges and Matthew Purver. 2006. Ro-
bust language analysis and generation for spoken di-
alogue systems (short paper). In Proceedings of the
ECAI 06 Workshop on the Development and Evalu-
ation of Robust Spoken Dialogue Systems.
Sebastian Varges. 2005. Chart generation using pro-
duction systems (short paper). In Proc. of 10th Eu-
ropean Workshop On Natural Language Generation.
Fuliang Weng, L. Cavedon, B. Raghunathan,
D. Mirkovic, H. Cheng, H. Schmidt, H. Bratt,
R. Mishra, S. Peters, L. Zhao, S. Upson, E. Shriberg,
and C. Bergmann. 2004. Developing a conversa-
tional dialogue system for cognitively overloaded
users. In Proceedings of the International Congress
on Intelligent Transportation Systems (ICSLP).
Fuliang Weng, Lawrence Cavedon, Badri Raghu-
nathan, Danilo Mirkovic, Ben Bei, Heather Pon-
Barry, Harry Bratt, Hua Cheng, Hauke Schmidt, Ro-
hit Mishra, Brian Lathrop, Qi Zhang, Tobias Schei-
deck, Kui Xu, Tess Hand-Bender, Stanley Peters,
Liz Shriberg, and Carsten Bergmann. 2005. A Flex-
ible Conversational Dialog System for MP3 Player.
In demo session of HLT-EMNLP 2005.
