Construct Algebra: Analytical Dialog Management 
Alicia Abella and Allen L. Gorin 
AT cT Labs Research 
180 Park Ave. Bldg 103 Florham Park, NJ 07932 
Abstract 
In this paper we describe a systematic 
approach for creating a dialog management 
system based on a Construct Algebra, a 
collection of relations and operations on 
a task representation. These relations 
and operations are analytical components 
for building higher level abstractions called 
dialog motivators. The dialog manager, con- 
sisting of a collection of dialog motivators, 
is entirely built using the Construct Algebra. 
1 INTRODUCTION 
The dialog manager described in this paper 
implements a novel approach to the problem 
of dialog management. There are three ma- 
jor contributions: the task knowledge repre- 
sentation, a Construct Algebra and a collec- 
tion of dialog motivators. The task knowl- 
edge representation exploits object-oriented 
paradigms. The dialog motivators provide 
the dialog manager with the dialog strate- 
gies that govern its behavior. The Construct 
Algebra provides the building blocks needed 
to create new dialog motivators and analyze 
them. 
The first main component of this dialog 
manager is the task knowledge representa- 
tion. The task knowledge is encoded in ob- 
jects. These objects form an inheritance hi- 
erarchy that defines the relationships that 
exists among these objects. The dialog man- 
ager exploits this inheritance hierarchy in de- 
termining what queries to pose to the user. 
No explicit states and transitions need to be 
defined using this framework (Bennacef et 
al., 1996; Meng and et. al., 1996; Sadek et 
al., 1996). A change to the dialog does not 
require a change to the dialog manager, but 
more simply, a change to the inheritance hi- 
erarchy. 
The second main component of this dia- 
log manager is the collection of dialog mo- 
tivators. The dialog motivators determine 
what actions need to be taken (e.g. ask a 
confirmation question). The dialog motiva- 
tors are founded on a theoretical framework 
called a Construct Algebra. The Construct 
Algebra allows a designer to add new moti- 
vators in a principled way. Creating a new 
application requires defining the inheritance 
hierarchy and perhaps additional dialog mo- 
tivators not encompassed in the existing col- 
lection. 
This dialog manager has been used for two 
applications. The first is a spoken dialog sys- 
tem that enables a user to respond to the 
open-ended prompt How may I help you? 
(HMIHY) (Gorin et al., 1997). The sys- 
tem recognizes the words the customer has 
said (Riccardi and Bangalore, 1998) and ex- 
tracts the meaning of these words (Wright 
et al., 1998) to determine what service 
they want, conducting a dialog (Abella and 
Gorin, 1997; Abella et al., 1996) to effec- 
tively engage the customer in a conversa- 
tion that will result in providing the service 
they requested. The second application is 
to Voice Post Query (VPQ) (Buntschuh et 
al., 1998) which provides spoken access to 
the information in large personnel database 
(> 120,000 entries). A user can ask for em- 
ployee information such as phone number, 
fax number, work location, or ask to call 
an employee. These applications are signifi- 
191 
cantly different but they both use the same 
dialog manager. 
2 Task Representation 
Information about the task is defined us- 
ing an object inheritance hierarchy. The in- 
heritance hierarchy defines the relationships 
that exist amongst the task knowledge. Ob- 
jects are defined to encode the hierarchy. 
This representation adheres to the princi- 
ples of object-oriented design as described 
in (Booch, 1994). Each of the objects has 
three partitions. The first partition contains 
the name of the object, the second contains 
a list of variables with associated values that 
are specific to the object, and the third par- 
tition contains any methods associated with 
the object. For simplicity of illustration we 
will not include any of the methods. Each 
of the objects inherits its methods from a 
higher level object called the Construct. The 
Construct's methods are the relations and 
operations that will be described in section 4. 
The result of the speech recognizer is 
sent to the spoken language understanding 
(SLU) module. The SLU module extracts 
the meaning of the user's utterance and pro- 
duces a list of possible objects with asso- 
ciated confidence scores that is interpreted 
by the dialog manager. The dialog manager 
then uses the inheritance hierarchy and an 
algorithm 1 fully described in (Abella and 
Gorin, 1997) to produce a set of semanti- 
cally consistent inputs to be used by the di- 
alog manager. The input is represented as 
a boolean expression of constructs extracted 
from the utterance. This input is then ma- 
nipulated by the dialog motivators to pro- 
duce an appropriate action, which most of- 
ten consists of playing a prompt to the user 
or generating a query to a database. 
3 The Construct 
A construct is the dialog 
knowledge representation 
manager's general 
vehicle. The task 
1An understanding of this algorithm is not nec- 
essary for the understanding of the work described 
in this paper. 
DIAL FOR ME 
:ORWARD NUMBER 
555-1234 
I 
BILLING 
NULL 
Figure 1: A construct example for HMIHY 
knowledge is encoded as a hierarchy of con- 
structs. The construct itself is represented as 
a tree structure which allows for the build- 
ing of a containment hierarchy. It consists 
of two parts, a head and a body. Figure 1 
illustrates a construct example for HMIHY. 
The DIAL_FOR_ME construct is the head 
and it has two constructs for its body, FOR- 
WARD_NUMBER and BILLING. These two 
constructs represent the two pieces of in- 
formation necessary to complete a call. If 
a user calls requesting to place a call it is 
the DIAL_FOR_ME construct that is created 
with the generic BILLING construct and 
the FORWARD_NUMBER construct with 
its value set to empty. The dialog manager 
will then ask for the forward number and for 
the type of billing method. In figure 1 the 
dialog manager has received a response to 
the forward number request. 
4 Construct Algebra 
The construct algebra defines a collection of 
elementary relations and operations on a set 
of constructs. These relations and opera- 
tions are then used to build the larger pro- 
cessing units that we call the dialog moti- 
vators. The set of dialog motivators defines 
the application. In this section we formally 
define these relations and operations. 
4.1 The Construct 
Definition 1 Head 
A head is an ordered pair <name, value>, 
where name belongs to some set of prede- 
192 
fined names, N, and value belongs to some 
set of predefined values, V. A value may be 
NULL (not assigned a value). 
Definition 2 Construct 
A construct is defined recursively as an or- 
dered pair <head, body> where body is a (pos- 
sibly empty) set of constructs. 
4.2 Relations 
The Construct Algebra defines six relations 
in the set of constructs. In each of the 
definitions, Cl and c2 are constructs. Note 
that the symbols C and C, introduced here, 
should not be understood in their usual 
"subset" and "proper subset" interpretation 
but will be described in definitions 4 and 5. 
Definition 3 Equality 
Two constructs are equal, denoted cl = c2 
when 
head(c1) = head(c2) and 
body(c1) = body(c2) 
Definition 3 requires that the heads of c1 
and c2 be equal. Recall that the head of a 
construct is an ordered pair <name, value> 
which means that their names and values 
must be equal. A value may be empty 
(NULL) and by definition be equal to any 
other value. The equality of bodies means 
that a bijective mapping exists from the 
body of cl into the body of c2 such that 
elements associated with this mapping are 
equal. 
Definition 4 Restriction 
Cl is a restriction of c2, denoted cl C c~, 
when 
head(c1) = head(c2) and 
(3f : body(c1) --+ body(c2))(fis 1 to 1 A 
(Vbl • body(cl))(bl C_ f(bl)) 
Intuitively, cl can be obtained by "pruning" 
elements of c2. The second part of the def- 
inition, (3f : ...) is what differentiates C 
from =. It is required that a mapping f be- 
tween the bodies of Cl and c2 exist with the 
following properties: 
\[ ~RSON 
Cl 
C 
< 
\ PERSON 
"",,,. ........... , 
ADD ~ES s 
STREET 
............................ ..- 
3H(}NE NUMBEI 
c2 
Figure 2: STREET and PHONE_NUMBER 
are "pruned" from c2 to obtain Cl. 
• f is 1 to 1. In other words, different 
elements of the body of O, call them hi, 
are associated with different elements of 
the body of c2, call them b2 
• The elements of the body of c1 are re- 
strictions of the elements of the body of 
c2. In other words, bl C_ b2, where bl are 
elements from the body of Cl and b2 are 
elements from the body of c2. 
Figure 2 illustrates an example. 
Definition 5 Containment 
cl is contained in c2, denoted Cl C c2, when 
Cl C_ c2 or (3b2 • body(c2))(Cl C 52) 
We assume that c1 C c2 either if Cl is a 
restriction of c2 or if Cl is contained in any 
element of the body of c2. Figure 3 gives 
an example. The AMBIGUITY construct 
represents the fact that the system is not 
sure whether the user has requested a 
COLLECT call or a CALLING_CARD call. 
This would trigger a clarifying question 
from the dialog manager. 
193 
? 
el 
C 
AMBIGUIT I 
'k ............. ""¢2 
~'ALLING_CARD 
CARD NUMBEI~ 
8485417 
Cl 
BILLING 
C2 
Figure 4: cj ¢--->c2 
Figure 3: cl C c2 
Definition 6 Generalization 
c2 is a generalization of el, denoted c1~__.~c2, 
when 
CALLING_CARD DIALFOR_ME 
head(cl)c--+head(c2) and 
(3f: body(c2) ~ body(c1)) 
(fis 1 to 1 A (Vba • body(c2)))(f(b2)~___b2) 
The generalization of heads means that 
the name of c2 is on the inheritance path 
of cl and their values are equal. Intuitively, 
c2 is an ancestor of Cl or in object-oriented 
C ~. terms ~C 1 is-a, 2 Note the similarity of 
this relation to C. Figure 4 illustrates an 
example. BILLING is a generalization of 
CALLING_CARD, or in other words CALL- 
ING_CARD is-a BILLING. 
Definition 7 Symmetric Generalization 
Cl is a symmetric generalization of c2, de- 
noted cl ~ c2, when 
C1¢--->C2 or c2¢---~Cl 
This definition simply removes the direction- 
ality of __¢---~. In other words, either 'tE 1 iS-a C2" 
194 
? 
CARD_NUMBER 
8485417 
BILLING 
Cl c2 
Figure 5: cl ¢--> c2 
or ;;c2 is-a c1" 
Definition 8 Containment Generalization 
Cl is a containment generalization of c2, de- 
noted ci ¢---> c2, when 
b2 is contained in c2 and cl is a symmet- 
ric generalization of b2. An example is illus- 
trated in figure 5. BILLING is contained in 
DIAL_FOR_ME and is a symmetric general- 
ization of CALLING_CARD. 
4.3 Operations 
The Construct Algebra consists of two 
operations union, U and projection, \. 
Definition 9 Union (U) 
We will define this operation in several 
steps. Each step is a progression towards a 
more general definition. 
Definition 9.1 Union of values (vl U v2) 
V 1 U V 2 = 
Vl, Vl = v2 and vl # NULL 
v2, Vl = v2 and Vl = NULL 
not defined, Vl # v2 
Recall that by definition, NULL is equal to 
any other value. 
Definition 9.2 Union of heads 
We define head(c1) U head(c2) only in the 
case c\] ¢-~c2, which is all that is needed for a 
definition of U. 
head(c I ) U head(c2) : 
value(el) U vatue(  )) 
Definition 9.3 (c, U c2) 
If c1~_~_c2, 
C 1 U C 2 = 
( head( c 1 ) U head(c2), 
u • body(  )} u 
{bllbl • body(c 1) A 
(Vb2 • body(c2))(bl #/(b2))}) 
In this definition the head of the resulting 
construct is the union of the heads of 
the operands. The body of the resulting 
construct consists of two parts. The first 
part is a set of unions (denoted f(b2) U b2 
in the definition above) where b2 spans the 
body of the second operand c2 and f is 
a mapping from Definition 6. Recall that 
the mapping f associates elements of the 
body(c1) with elements of the body(c2) such 
that f(b2)~-+b2 for b2 • body(c2) so the 
union f(bj U b2 is (recursively) defined in 
Definition 9.3. The second part of the body 
of the resulting construct consists of those 
elements bl of the body(c1) that no element 
from the body(c2)maps into through the 
mapping f. In other words, the second part 
of the body consists of those elements "left 
CALLIN 
¢ 
CARD-NUMB 1 
NULL u 
EXP|RATIO~ 
__ 299 / 
Cl 
_CARD :ALLI~ 
1 CARD NlYMB~, 
1239834 = 
c2 
Figure 6: cl U c2 if c1¢-.-~c2 
LLINO_CARD 
~ ARD_NUMBER 1239834 
EXPIRATIO1 ~ 
299 
behind" in the body(cl) after the mapping 
f. Figure 6 illustrates an example. The 
union operations results in a construct 
with the head CALLING_CARD and a 
body that contains both CARD_NUMBER 
and EXPIRATION. The CARD_NUMBER 
construct from Cl and c2 can be combined 
because the value of CARD__NUMBER from 
cl is NULL. The construct EXPIRATION 
is added because it does not exist on the 
body of c2. 
Definition 9.4 Cl U c2 
If C 1 ,-v C2, 
ciUc2, ci ~-+c2 
C 1 U ¢2 = C 2 U el, C2 ~ C1 
Definition 9.5 cl U c2 
If cl ~-+ c2, 
C 1 U c 2 = 
C 1 U c2, 
(head(c2), 
{el U b~lb~ • body(c2) A cl ~ b2}U 
{b2152 • body(c2) ^ Cl b£), 
C1 ,"-' C2 
C1 ~ C2 
Figure 7 illustrates this union. The head 
of the resulting construct is the head of 
c2 which is DIAL_FOR_ME. The resulting 
construct no longer has BILLING but 
195 
:ALLING CARD 
EXPIRATION 
AL 
~ORWARD~NUMB FZ~ 
BILLING \[ 
Cl C2 
DIAL_FOR_ME 
~LLING_CARD 
ARD_NUMBEI 
EXPIRATION ! 
Figure 7: Cl I.J C2 if cl ~ c2 
rather CALLING_CARD since BILLING is 
a generalization of CALLING_CARD. In 
addition the resulting construct contains the 
construct FORWARD_NUMBER because it 
remains from DIAL_FOR_ME. 
Definition 9.6 Cl U e2 
In the general case, 
C1 ~ C2 -~- 
el \[,-J e2, 
c2 \[..J Cl, 
((REP, NULL), {cl, c2}), 
C1 ~ C2 
e2 ~ el 
Cl ~ C2 and 
C2 ~ Cl 
In this definition REP is a construct used to 
represent the union of those constructs that 
do not satisfy any of the aforementioned 
conditions. By definition REP has a value 
of NULL and the body consists of the 
constructs Cl and e2. 
Definition 10 Projection (\) 
CI\C 2 ~-. 
((AMBIGUITY, NULL), 
{hi U c2161 C c1 A bl ~- c2}) e2 ¢-+ cl 
Cl C2 ~ el 
Figure 8 illustrates an example of an am- 
biguous construct and the result of the 
FIRST NA\] 
C2 C1\C2 
Figure 8: Projection operation example 
projection operation. The construct is 
AMBIGUITY because all the elements of 
its body have the value of 6151 for DEPT. 
In this example, c2 contains the construct 
LAST_NAME with the value of Smith. 
There are 2 constructs on the body of Cl 
that are in the relation b2 C Cl, in other 
words have value for LAST_NAME of Smith. 
Therefore the result is an AMBIGUITY 
construct with two elements on its body, 
both with the LAST_NAME value of Smith. 
5 Dialog Motivators 
A dialog motivator determines what action 
the dialog manager needs to take in con- 
ducting its dialog with a user. The di- 
alog manager for HMIHY currently con- 
sists of 5 dialog motivators. They are dis- 
ambiguation , confirmation, error handling 
(recovery from misrecognition or misunder- 
standing and silence), missing information 
and context switching. VPQ uses two addi- 
tional motivators, they are continuation and 
196 
co: Construct used for disambiguation, 
cQ Ec 
CA: User response 
Dk(c, cigK) = 
c, c ~ AMBIGUITY 
Dk+l (c, CIDK), CA ~__~_ERROR 
Dk+l (C, CID g (.J CQ), c A   IDK 
C\CA, C A ¢-----} C 
C A C A ~ C 
Figure 9: Disambiguation Motivator 
database querying. 
The disambiguation motivator determines 
when there is ambiguous semantic informa- 
tion, like conflicting billing methods. Con- 
firmation is used when the SLU returns a 
result with low confidence. Error handling 
takes on three forms. There is error recovery 
when the speech recognizer has likely misrec- 
ognized what the user has said (low confi- 
dence scores associated with the recognition 
results), when the user falls silent, and when 
the user says something the SLU does not 
expect or does not handle. Missing infor- 
mation determines what information to ask 
about in order to complete a transaction. 
Context switching is the ability of the sys- 
tem to realize when the user has changed 
his/her mind or realizes that it has mis- 
understood and allows the user to correct 
it. The continuation motivator determines 
when it is valid to offer the user the choice to 
query the system for additional information. 
Database querying decides when the system 
has acquired enough information to query a 
database for the requested information. 
5.1 Disambiguation Motivator 
Figure 9 illustrate how the disambiguation 
motivator is created using the Construct 
Algebra. The disambiguation motivator is 
called with the current construct c and a 
set of constructs called CID g that represents 
information that the user does not know 
(IDK - "I Don't Know"), in other words, 
the user explicitly responds to a prompt with 
the phrase "I don't know" or its equivalent s. 
2The phrases chosen are based on trials 
Input: A sequence of semantic input from 
the SLU module in response to a prompt 
Output: Complete construct c (no need for 
further dialog) 
Repeat 
For all dialog motivators DMI 
if DMi applies to c 
Perform action(DMi,c) 
Apply Dialog Manager to get CA 
Using Construct Algebra, 
combine c and CA into c 
Until no motivator applies 
Return c 
Figure 10: Dialog Manager algorithm 
The motivator runs through several checks 
on the construct c. The first is to check to see 
if in fact the motivator applies, or in other 
words if c is a restriction of AMBIGUITY. 
If it is not then the motivator simply return 
c without changing it. The second step is 
to check to see if the ERROR construct is a 
generalization of CA where CA represents the 
user's response. The ERROR construct rep- 
resents an error condition like silence or mis- 
recognition. If it is, then it goes on to next 
motivator because this motivator does not 
apply to error conditions. If CA equals the 
IDK construct then this means that the user 
did not know the answer to our query and we 
add the construct used for disambiguation, 
cQ to the set of constructs ¢IDK. If however, 
CA is in the containment generalization rela- 
tion with c then the projection operation is 
applied and the result is returned. If CA is 
not in this relation then this indicates a con- 
text switch on the part of the user and the 
disambiguation motivator returns CA as the 
result. 
All other motivators are constructed in a 
similar fashion. An application can use these 
motivators or create new ones that are ap- 
plication specific using the operations and 
relations of the Construct Algebra. 
197 
System" VPQ. What can I do for you? 
User: I need the phone number for Klein. 
System- I have more than 20 listings for 
Klein. Can you please say the first name? 
User: William. 
System" I have 2 listings for William Klein. 
Can you tell me the person's work location? 
User: Bedminster 
System" The phone number for William 
Klein is 973 345 5432. Would you like more 
information? 
User: No. 
System" Thank you for using VPQ. 
Figure 11: A sample dialog for VPQ 
6 Dialog Manager 
The input to the dialog manager is a collec- 
tion of semantic input generated by the SLU. 
Figure 10 illustrates the algorithm used by 
the dialog manager. The output is the com- 
plete construct c which no longer requires 
further dialog. The algorithm loops through 
all the dialog motivators determining which 
one needs to be applied to c. If it finds a mo- 
tivator that applies then it will perform the 
necessary action (e.g. play a prompt or do 
a database lookup). The algorithm repeats 
itself to obtain CA (the construct answer). In 
other words, the construct that results from 
the action is subject to the dialog motiva- 
tors starting from the beginning. Once CA 
has been found to be complete it is combined 
with c using Construct Algebra to produce 
a new construct. This new construct c also 
goes through the loop of dialog motivators 
and the procedure continues until no moti- 
vator applies and the algorithm returns the 
final construct c. 
6.1 Example 
To illustrate how the dialog manager func- 
tions we will use an example from VPQ. 
Figure 11 illustrates a sample dialog with 
the system. The sequence of motivators for VPQ 
is error handling, confirmation, miss- 
ing information, database querying and dis- 
ambiguation. The construct that is created 
as a result of the user's initial utterance 
is shown in figure 12. All the information 
needed to do a database lookup is found in 
the user's utterance, namely the piece of in- 
formation the user is seeking and the name 
of the person. Therefore the first motivator 
that applies is database querying. This moti- 
vator creates the database query and based 
on the result creates the construct CA. The 
construct CA is then searched by each of the 
motivators beginning again with error han- 
dling. The motivator that applies to CA is 
the disambiguation motivator because there 
are more than 20 people in the database 
whose last name is pronounced Klein, in- 
cluding Klein, Cline and Kline. The dis- 
ambiguation motivator searches through CA 
to determine, based on preset parameters, 
which piece of information is most useful for 
the disambiguation process as well as which 
piece of information the user is likely to 
know, which is selected when the inheritance 
hierarchy is designed. For VPQ this includes 
asking about the first name and work loca- 
tion. In this example the dialog manager 
searches the database entries and determines 
that the most discriminating piece of infor- 
mation is the first name. Once the user re- 
sponds with the first name there are still 2 
possible candidates and it asks for the next 
piece of information which is work location. 
Had the user not known the work location 
the system would have read out the phone 
number of both people since the total num- 
ber of matches is less than 3. If the num- 
ber of entries after disambiguation remains 
greater than 3 the system refers the user to 
a live operator during work hours. 
7 Conclusion 
In this paper we have described a novel ap- 
proach to dialog management. The task 
knowledge representation defined intuitively 
and without the need to define call flows in 
the traditional finite-state approach. The 
Construct Algebra serves as the building 
blocks from which the dialog motivators 
that drive the dialog system are comprised. 
Building a new application will only require 
the designer to define the objects (e.g. COL- 
198 
Figure 12: Sample construct for VPQ. 
LECT, CREDIT etc.) and the inheritance 
hierarchy. The Construct Algebra serves as 
an analytical tool that allows the dialog mo- 
tivators to be formally defined and analyzed 
and provides an abstraction hierarchy that 
hides the low-level details of the implemen- 
tation and pieces together the dialog motiva- 
tors. This same dialog manager is currently 
being used by two very different applications 
(HMIHY and VPQ). 

References 
Alicia Abella and Allen L. Gorin. 1997. 
Generating semantically consistent inputs 
to a dialog manager. In Proc. EuroSpeech 
Rhodes, Greece. 
A. Abella, M. K. Brown, and B. Buntschuh. 
1996. Development principles for dialog- 
based interfaces. European Conference on 
Artificial Intelligence. 
S. Bennacef, L. Devillers, S. Rosset, and 
L. Lamel. 1996. Dialog in the rail- 
tel telephone-based system. International 
Conference on Spoken Language Process- 
ing. 
Grady Booch. 1994. Object-Oriented Anal- 
ysis and Design with Applications. Ben- 
jamin Cummings. 
B. Buntschuh, C. Kamm, G. DiFabbrizio, 
A. Abella, M. Mohri, S. Narayan, I. Zelj- 
vokic, R.D. Sharp, J. Wright, S. Marcus, 
J. Shaffer, R. Duncan, and J.G. Wilpon. 
1998. VPQ: A spoken language interface 
to large scale directory information. In 
Proc. ICSLP Sydney. 
A.L. Gorin, G. Riccardi, and J.H. Wright. 
1997. How May I Help You? Speech Com- 
munciation. 
Helen Meng and Senis Busayapongchai et. 
al. 1996. Wheels: A conversational sys- 
tem in the automobile classifieds domain. 
International Conference on Spoken Lan- 
guage Processing. 
G. Riccardi and S. Bangalore. 1998. Au- 
tomatic acquisision of phrase grammars 
for stochastic language modeling. In Proc. 
ACL Workshop on Very Large Corpora, 
Montreal. 
M.D. Sadek, A. Ferrieux, A. Cozannet, 
P. Bretier, F. Panaget, and J. Simonin. 
1996. Effective Human-Computer Co- 
operative Spoken Dialogue: the AGS 
Demonstrator. International Conference 
on Spoken Language Processing. 
Jerry Wright, Allen L. Gorin, and Alicia 
Abella. 1998. Spoken language under- 
standing within dialogs using a graphical 
model of task structure. In Proc. ICSLP 
Sydney. 
