Maintaining Consistency and Plausibility in Integra,ted Natm-al 
Langu~ge Understanding 
'" "" :('amad~_, Toyoaki Nishida, Xuemin Liu, c>hup Doshita,, and Atsuahi ~ ' :a 
Department of hdbrmation Science 
Kyoto University 
Sa,kyo-ku, Kyoto 606, Japan. 
phone: 81.~.75,.751-2111 ext. 5396 
email: nishid~%doshit~Lkuis.kyoto-u.j,met %j apan@relay.cs.net 
Abstract 
In this paper, we present an inference mecha- 
nism called the integrated parsing engine which 
provides a uniform abductive inference mech-- 
anism for natu~al language understmlding. It 
can (1) make phmsibh, assmnptiox~s, (2) rea~. 
son with mlfltiple alternatives, (3) switch the 
search process to the maximally plausible alter.., 
native, (4) detect contradiction attd tame COlt -~ 
clutions which depend on inconsistent a,'~sump- 
tions, and (5) update plausibility factor of each 
befief based on new obsexvations. We demon- 
strafe that a .natural language understanding 
system using the integrated parsing engine as a 
subsystem can pursue a guided search for most 
t)lausible interpretation by making use of syn- 
tax, semantics, and contextual information. 
1 Introduction 
Natural language understanding involves lots of 
hard issues such as various types of ambiguities, 
indeterrtfinacies caused by ellipses or fragmental 
utterances, or ill-formedness. Being confronted 
with these difl\]culties, it does not seen, reason-- 
able toseek for a method of logically deducing 
the spe~ker's intexMed meaning or p\]an from 
utterances. Insteaxt, it is much more natural to 
characterize natural language understandhlg a~q 
an abductive process of exploring most plausb 
ble interpretation which can ext)lain given ut- 
t el'a/Ices. 
In this paper, we present an abductive in-. 
ference mechanism, called the integrMcd pars- 
ing engine, for natural lmlguage lmdcrst;mding. 
The integrated pea'sing engine is ~ble to: 
make plausible assunrptions at z,:pproprlate 
time 
reason with multiple alternatives based o~ 
ditferent sets of a~ss~m~ptions 
switch the sem'ch process to the maximally 
plausible alternative 
, detect contradiction resulting from inco~v 
sistent ~sumpfions and eliminate ~fil con- 
clutions which depends ,',m these assump~ 
tions 
* update plausibility factor of each belief 
based on new observations. 
Thus, the integrated parsing engine is generM 
enough to carry out hngulstic and nonlingulso. 
tic inferences in a uniform manner, by drawing 
information from various sources: syntax, seo 
mantle, discourse, pragmatics, or real world. 
In the remainder of this paper, we first de-~ 
scribe mechanisms for maintaining consistency 
and plausibility. We then show how these two 
mechauisms interact to guide the inference pro.. 
tess. Finally, we use an implemented exam= 
ple to demonstrate how the integrated parsing 
engine is used to interpret sentences by taking 
contextual factors into account. 
2 Maintaining Cons~si, e:c~cy 
The CME (Consistency Maintenm~ce Engixtc) 
is a component of the httegrated pa:rsmg (mghte 
4B2 
re~;ponsib\]e fl)r maintaining consistency among 
beliefs. Basic design principles of the CME 
is b~L, md on de Kleer's ATMS (Assumption- 
bz~sed '\]!i'uth Maintenance Engine) \[de 86\]. 
The CME maintains a set of alternative be- 
tlet~ eae~ of which consists of a set of as- 
~mp~ion~ m~d their conclusions, as follows: 
alter'na~,ive I {./ilj~o°,,A!,rt~} Bll~ .... ,Blm~ 
alternative re (A,,i~. • . ,Aura,, } Bul~ ...~ Bunt,, 
en vlr o~tme~t couclu sious 
An extc:c~fl problem solver is assumed to exist 
which makeu a~sumpfion, adds conclusion, and 
dctcd;s contx~a~li(:tion° 
~\['he mv~n ~ask of CME is to maintain alterna- 
tive bc~i('2~ by removing all alternatives whose 
:;ct of a:~'~mmptions has turned out contradic- 
tory° Lik(, ATMS, the CME takes advantage of 
the followi~,g monotonic property: 
if ~ contr~dictlo** is derived from a set 
of assumptions A, then contradiction 
is Mso derived from any set of assump- 
tions B such that B D A. 
E~={A,} E2={Au} E. ={A=} 
Ell El., E~I E~m. 
={AI,Au} ={A1, A,.,} ={A.,A.1) ={A~,A .... } 
Figure 1: The E-tree 
E1 :: {AI} E2 = {A2} E. = {A.} 
P 11///// ~/) 1 n I Pn~ ~Pu m . 
En El,,~ E.I E.,.. 
= {AbAn} ={A1,A,.,} ={A.,A.I} = {A.,A.,..} 
Figure 2: The E-tree with Conditional Proba- 
bilities 
Thus, if contradiction is derived from a set 
of as.,mm~)tions { t~-~, D ), alternative in terpreta- 
tiol~s depending on sets of assumptions such as 
{B,C,D}, {A,B,D}, \[A,B,C,D}, ... are re- 
moved. \[n addition~ t, he GME keeps records 
of contradictory sets of assumptions to prevent 
any interpretation depending on them from be- 
ing considered in future. 
Unlike ATMS whose control regime is bread- 
first, our CME uses a tree called the envh'on- 
ment tree, or the E-tree for short, to guide the 
search process. Each node of the E-tree rep- 
resents an environment, a set of assumptions. 
\]i;alch arc of the E4ree represents that a lower 
node is derived from the upper node by mak- 
ing one :more assumption. Thus in figure 1, E0 
is the root node, and it represents an environ- 
mnet without any assumption. Nodes below 
-5;0 :represent environments with one or more 
assumption added to its parent node's envi- 
:r,~x~meaL Thus, El :: E0 U {A1} = {A,}, 
~:_~1 := J\[!;:, U (AH} =: (AI,AH}, and so on. 
We assume that a set of assumptions made at 
~he same parent node axe mutually exclusive. 
Although this is a rather strong assumption, 
it, makes sense in ~tatural language tmderstand- 
ing :~ince many assmuptions being made dur- 
i~g the natural language mlderstanding process 
are mutuMly exclusive. Even if this is not the 
c~se, any set of assumptions can be transformed 
into a set of mutually exclusive assumptions by 
adding appropriate conditions. Although this is 
a cumbersome solution, it does not often take 
place in natural language understanding and 
most importantly it saves tile amottnt of com- 
putation. 
Note that the CME alone cannot determine 
which way to go when there is more than one 
possibility of extending the set of beliefs. This 
information is provided by the PME, as de- 
scribed in the next section. 
3 Maintaining Plausibility 
The PME (Plausibility Maintenance Engine) 
inaintains estimations of how plausible each en- 
vironment is. This information is given as con- 
ditional probabilities and it is kept as annota- 
tions to each arc of the F,-tree. Thus, in figure 2, 
which is a slightly more precise version of fig- 
ure 1, Pl stands for P(EI), pq for P(EjIAi), 
pi./~ for P(Ek, IAi, Aj), etc. 
It follows from the property of conditional 
probability that 
= O, 
if i ~ j and El and Ej are immediate children 
4133 
(a) initial E-tree. (b) The F~tree after -,E~ is 
observed. 
~'0 E0 
Ex I);= E1 I-';2 
t> I i o "-... 
Ea E4 Es Ea 1/)4 Es 
Figure 3: A Sample E-tree with Annotation 
of the same parent. Furthermore, 
if Ej is a parent node of Ei. 
Initial value of pi's are to be given from the 
external problem solver. The PME's role is to 
maintain estimation of prausibility by taking 
into account given observations. Currently we 
only take -~E, the event of environment E run- 
ning into contradiction, as an observation. We 
use a Bayes' law to modify P(A) into P(AI-E). 
Thus, 
P(~EilE~ ) • P(E~) 
(1- P(EjlP~)). P(E,).(1 ) 
1- P(Ej) 
if El and Ej are brothers, (1) is further simpli- 
fied to: 
P(E,) 
1- P(Ej)" (2) 
For example, suppose it has turned out that 
environment E4 is in contradiction and hence 
-E4 is observed (figure 3(a)). The annotations 
to the E-tree are updated as in figure 3(b). 
Notice that the update of conditional proba- 
bility can be done based on local information. 
Linl uistic and Nonlingaistic Pwblem Solve~ 
I Working Memory 
Knowledge Base 
Associative Networks 
Problem Solving Engine Previmm Topic (PSi) 
The Integrated Parsing Engine \[ E-tn:e (CME) .~ E0 
,.~ ............................. E 1 Plauaibi|ity M~intenance Engine ~'~ E~ 
Figure 4: The Structure of a, Natural Lan.. 
guage Understanding System with the Inte- 
grated Parsing Engine as a subsystem 
4 Natural Language Un- 
derstanding System Uso 
ing the Integrated Pars  
lag Engine as a Subsys  
tern 
The integrated parsing engine consists of the 
CME and the PME. The architecture of a natu- 
ral language understanding system with the in- 
tegrated parsing engine as a subsystem is shown 
in figure 4. 
The knowledge base contains various types 
of information for language comprehension, in- 
cluding lexicon, morphology, syntax, semantics, 
discourse, pragmatics, commonsenses, and so 
on. The whole system is controled by the prob- 
lem solving engine (PSE). The PSE can access 
to the knowledge base and use the integrated 
parsing engine as an aid to seek for most plau- 
sible interpretation. Input texts are analyzed in 
a sentence-by-sentence manner. The discourse 
structure is maintained as a previous topic in 
the working memory° 
When it scans a new sentence, the PSE tirs~ 
initialize the F~tree with only the root node° 
Then the PSE repeats the following cycle: 
(step 1) choose a leaf node with the high- 
est probability as a working envirb 
oment 
(step 2) repeatedly derive conclusions h'om 
4~34 
the |ibrary the xerox the irmeting 
rooln rool~l l'oolll 
-. I t // 
"\ I ! keyI key2 hey3 
l~4x 
....... (* ...... ~' . ..... 
,. o ° 
KA ¢3I WO I(A S\]I I '1'i,~ KU DA SA \]'~k 
~'" ../the) key <object> lend .... Id y ..... ": .) 
the 1.'ojessor 
: ~gure 5: o,mq~le Dialog Environment 
believed p~'opositions until either (a) 
the goal is achieved, (b) contra, 
diction is derived~ or (c) no more 
conchlsion is derived ~mless making 
more assumption. 
In case (a), the process hMts. 
In case (b), the process is passed to 
the PME~ which modifies current es- 
timation of plausibility so that this 
f,~:t is reflected, then nat alternative 
of meximum plausibility is dmsen 
~(l is suggested to the CME. 
In case (c)~ the process also is passed 
to the PME, which assigns plausibib 
ity to new nodes, and working cnvi- 
ronment is chosen agMn. 
The integrated parsing engine has been writ- 
ten in Lisp. It is running with a small exmerl- 
mental grammar for Japanese. The next section 
shows how it works. 
5 .A.n Example 
Suppose a dialog envh'omne~tt in which a pro.. 
fesso~" speaks to a clerk to borrow a key of 
some rooms (figure 5) and utters the following 
Ja\]pa~e~e ~extteltce: 
(3) KA GI WO KA SH ITE KU DA SA I 
(~/the) key <object> lend could you ...? 
"co,ad you le,,d (me) (a/,h,:) key." 
L/a .../"-,., ~./a 
{~word-1} {~.,~o~d-~, } 
Figure 6: I);~rec after assumptions @word-1 
and @word-2 are made 
'i'he referential meaning of this sentence is 
ambiguous if there is more than one key in a 
given situation. Suppose three keys are there: 
key1 for a hbrary room, key2 for a xerox room, 
and key3 for a meeting room. 
Although sentence (3) is ambiguous in nor- 
real contexts, it becomes much lcss so if it fol- 
lows sentences like: 
(4) HO N WO KO PI I SHI TA I NO DE SU GA 
"I'd like to xerox some books." 
Even if no previous sentence is spoken, sen-. 
fence (3) is acceptable in a situation where the 
speaker and the hearer rmltually believe that 
the xerox room is accessed so often that "the 
key" is usually uscd to refer to key& the one 
for the xerox room. 
Note that the omission of the patient case 
does not matter in usual situations, since there 
is a strong defa~flt that the filler of this case is 
the speaker. 
Now let us show how sentence (3) is ana- 
lyzed in a context where sentence (4) was pre- 
viously uttered. The task of analyzing input 
starts from recognizing words. Lots of ambi- 
guities arise in this phase. For sentence (3), 
'KA' might be a single word 'KA' (postposi- 
tion marking interrogative) or a part of a longer 
word 'KAGI' (key). Since longer match is con- 
sidered to be more plausible in generM case in 
Japanese analysis, we assign larger number of 
probability to the latter possibility. Following 
this anMysis, the PSE makes the assumptions 
to the integrated parsing engine: 
@toord-1 (t~ke the sequence ~KA t as a word): 
~-~ probability 1/3. 
@word-2 (ta&e the sequence 'KAGI' as a word): 
probability 2/3. 
Accordingly, 'the CME extends the initial E-tree 
as in figure 6. Since, the enviromnent E1 has 
the highest plausibility, the CME chooses it for 
the next environment and control is returned to 
the PSE. 
4~3 5 
k , the libra W cy t ............ room ,,,,., . 
book 
ke~2 ........ I, ohg,le:y_~__ ~. 
×eroxing 
hey3 ....... the meeting--meeting 1"0 0I~1 
Figure 7: A.n Associative Network between 
Concepts 
Now the PSE tries to derive further conclu- 
sion in the chosen environment. After having 
i'ccognized that the pm't of speech of the word 
'KA(~I' i~ noun, the PSE tries to find out the 
referent of the noun and reahzes that thi'ee am- 
bigtAties arise lit this situation. Again, the PSE 
calls the CME to make assumptions. At the 
same time, the PSE is called for to assign esti- 
mated conditional probabihties to each assump- 
tion° 
Currently, the system uses an associative net- 
work as shown in figure 7 to determine plausL 
bility. Nodes of this network represent either a 
concept or art instzatce, and arcs mean that the 
two concepts or instants at its both ends have a 
certain relation. Those items which have dense 
conuections to previous subjects are considered 
to be plausible as a referent. In our example, 
since the node xerox is marked as the previous 
subject key2 is considered most plausible, while 
key1 is less plausible and key3 much less. Thus, 
the following assumptions are made: 1 
@re fereni-1 (consider 'KAGI' to refer to keyl): 
=~ probabiliy 1/3. 
@referent-2 (consider 'KAGI' to refer to key$): 
--~ probabiliy 1/2. 
@re\[erenl-3 (consider 'KAGI' to refer to key3): 
=ez probabiliy 1/6. 
In case no previous utterance is given, the 
PSE will consult information given as a priori 
measurements. 
The E-4ree now becomes as in figure 8, a~td 
{@word-2, @referent-2}, which is the most 
1 Currently we use a very simple algorithm for assign- 
ing those value: when there are three alternatives, the 
densest connection receives the vMue (1/3), the second 
(1/2), and the third (1\]6), regardless of how closely they 
are related to each other. We plan to develop a much 
more precise method in a near future. 
E~ 
1/3 ~ ~.~. 2/3 
{Qwo,.d-1} {~wor<l-2} 
{@word-2, {@word-2, {@word-?,, 
@referent- 1} @referenb2} @referenb3} 
Figure 8: E-.tree after assumptions about {,}L~: 
referent of 'KAGI' (key) are made 
meaning-2 meaniag-3 meanings4 
,o,o,oot-,4 ,o,o,o,.-,A tO--=o--nt-=Xl*o=o==-;J .! /I 
noun 1 .t- post-I verl> l 
" l 1 .... .d-4 1 
{-,o .... ,_, A ooo,.=-=l \-o.o,=.=/I 
ch-I ch-2 ch-3 ch-4 ch-5 °'~ ch-lO 
I I I I I t 
KA GI WO KA StlI T\]~ I(U DA SAI 
Notice that ~1l part of this netwm'k is not explored in 
actual processing. 
Figure 9: Dependency of Befiei~ 
plausible enviromnent at tiffs point, is chosen as 
the next environment. The analysis is contin- 
ued this way until the semantic representation 
is obtained for the whole sentence. The inter- 
pretation obtained tlds case is: 
event = asking-for \] 
actor = <the speaker> 
object = key2 
Figure 9 shows the dependency structtu'e of be- 
fiefs related to this analysis. 
Notice that the efficiency of the analysis is 
significantly improved when strong expectation 
exists. For example, although character 'sin' h~ 
sentence (3) has many possible interpretations 
in Japanese, the system is not annoyed by those 
ambiguities, since this part of the sentence just 
goes as expected. The system may come to sus- 
pect it only when most of its expectation faik. 
G 
{@word..,l\] {@word-2} 
{@word-2, {@word-2, {@word-2, 
Co)referent~ 1} (c~referent- 2 } @referent-3} 
addition~ the integrated paxsign engine provides 
a concise and high level mechatdsm for abduc~ 
tire reasoning. We have carefully chosen a set 
of reasonably high-level functions necessary for 
abductive reasoning. This serves to much sim- 
plifying natur~ langu.age mtdersta~tding system 
than otherwise. 
li'ig,~re 10: Gtree after assumptions about the 
proposed interpretation based on {@word-2, 
@referent-.2} is rejected 
Now suppose the above interpretation is re- 
jected for some ~'eason, say by expficitly negated 
by the speaker. Th.e~ the system will eventu- 
ally produce an alte~atative interpretation tak- 
ing key1 as a referent, by changing ammtations 
to the E4ree as lit figm'e 10. 
6 Related Work 
This paper was inspired by a number of works. 
A massively par-Mlel parsing by Waltz and 
Poll~k \[WP85\] has demonstrated the etfect 
of integration through a uniform computa- 
tion me(hanism (marker passing) in context- 
dependent comprehension of discourse. They 
have pointed out the importance of non-logical, 
associative relation between concepts. Char- 
niak has pointed out the abductive nature of 
language comprehension. Chat'niak's Wimp 
\[Cha86\] uses a marker passing mechanism as 
a basis of abductive inference engine for lan- 
guage comprehension. But it is not used alone; 
it is augmented by a logical process called path 
proof. \[na parser used in Lytinen's MOP-. 
TITANS \[Lyt86\], a mechanism is provided to 
allow close interaction between syntax and se- 
mantics, while keeping the modularity of the 
system. Another thing to note is that Lytinen's 
integrated parser makes use of strong semantic 
expectation to constrain the search. 
The integrated parsing engine presented in 
this paper takes advantages of these preced- 
ing works. Unlike Waltz and Pollack, and like 
Charniak and Lytinen, our integrated parsing 
engine has a hybrid architecture for logical atld 
non-logical inferences. What is novel with ore" 
integrated pat'sing engine is the method of inte- 
grating and maintaining logical and non-logical 
~nformafion Obtained from various sottrce. In 
7 Concluding Remarks 
We have presented an inference engine for inte- 
grated natural language understanding, based 
on a characterization of natural language un~ 
dcrstanding as an abductive process. The 
essence of our approach is connecting con- 
sistency maintenance engine and plausibility 
maintenance engine closely enough to allow 
their dense interaction. Although we have 
shown rather "low level" issues, we believe the 
same idea is applicable to "higher level" prob- 
lems such as inferring speaker's intention and 
plan. 

References 

Eugine Charniak. A neat theory of 
marker passing. In Proceedings AAAL 
86, pages 584-588~ 1986. 

Johan de Kleer. An assumption-based 
tins. Artificial Intelligence, 28:127-- 
162, 1986. 

Steven Lytinen. Dynamically combin- 
ing syntax and semantics in nabn'M 
language processing. In Proceedings 
AAAI-86, pages 574-578~ 1986. 

D. Waltz and J. B. Pollack. Massively 
parallel parsing: a strongly interactive 
model of natural langqlage interpreta- 
tion. Cognitive Science, 9:51-74, 1985. 
