A Project Report on NP: an Assumption.based 
NL Plan Inference System that uses Feature Structures 
John K. Myers 
ATR Interpreting Telephony Research Laboratories 
Sanpeidani, Inuidani, Seika-cho, Soraku-gun, Kyoto 619-02, Japan 
myers@atr-la.atr.co.jp 
Abstract 
This paper presents a project report on NP, a working Natural 
language Plan inference system that uses feature structures 
and is based on assumptions. Input to the system is in the 
form of feature structures, which can be taken directly from 
the output of a semantic parser. Plan actions are represented 
by feature-structure plan schemata with preconditions, hier- 
arclfical decompositions, and effects. Output is in the form of 
a network of believed assertions represented in a knowledge 
base, and can be reported, used to answer generation-system 
queries, or drive side-effecting demons. The plan inference 
component is implemented using a feature-structure-based in- 
ference engine and models of plan recognition, prediction, and 
inference. The inference engine is implemented using a rewrit- 
ing system for patteru-matctfing, and an Assumption-based 
Truth Maintenance System (ATMS) for conjunctions. The 
ArMS allows pre-instantiation of hypothetically known as- 
sertions and implications, which can significantly reduce pro- 
cessing time. The ATMS also permits simultaneous consider- 
ation of multiple possible inputs or multiple possible inferred 
plan outputs; these can be mutually conflicting or supportive. 
This capability will be important for disambiguation. The NP 
system is used to infer dialog- and domaln-level plans, among 
other types. 
Original contributions include: a plan inference system 
that works directly from feature structures; a plan inference 
system that uses an ATMS and plan schema actions with pre- 
conditions and effects to infer hierarchical and chained plans; 
and, an inference engine that works with multiple feature- 
structure assertions and rules. 
Project Goal. This project is aimed toward 
a dialog understanding system that can be used as 
part of an automatic interpreting telephone system. 
Interpretation will be performed by parsing, trans- 
ferring, and generating utterances. Thus, dialog un- 
derstanding will be used to recognize speech acts and 
illocutionary acts, resolve ellipses, and provide re- 
quired missing information, among other tasks. The 
understanding system will use the output of the se- 
mantic parser, and provide information to the trans- 
fer module and generation system. Therefore, fea- 
ture structures should be used as the basic data 
representation scheme. Dialog understanding re- 
quires a general-purpose plan inference engine that 
can work with dialog plans, domain plans, common- 
sense knowledge plans, and so forth. The system 
must also in the future be able to perform disam- 
biguation of possible utterances. 
Background: Assumptions. The plans, in- 
tentions, beliefs, etc., of a human are menial con- 
cepts which cannot be perceived directly', they are 
unobservable\[Mye88\]. There is insufficient infor- 
mation to represent these concepts with certainty. 
Therefore, the system must be able to represent con- 
cepts in an uncertain manner, using assumptions. 
Communication between two people is inherently 
an assumption-based process. Since it is never corn- 
pletely possible to directly know the concepts of an- 
other person, it is necessary in the course of a con- 
versation to take a stance and rely on assumptions 
about the other person's thoughts \[Den87\]. Thus, in 
a dialog understanding system, there are at least two 
kinds of assumptions that must be represented: as- 
sumptions that the two speakers make, which must 
be modeled by the system, and assumptions that the 
system makes about the situation, 1 the two speakers 
and their plans, intentions, etc. 
Design. Plan inference and other knowledge- 
based reasoning tasks require that multiple conjunc- 
tive implications be matched against large sets of 
unordered assertions. The system will have a cat- 
alog of world knowledge, common-sense knowledge, 
and assertions which are believed by the speakers. 
These must be accessed non-sequentially and used 
for reasoning. In other words, language understand- 
ing should be done by using an "expert system" in- 
ference engine, 
Computer languages should be used according to 
their strengths and weaknesses. Feature-structure 
systems are strong in representing complex, incom- 
plete, or underspecified information, and in perform- 
ing unification. However, they are extremely ineffi- 
cient at list processing and numeric calculations (e.g. 
for evidential reasoning), and don't represent multi- 
ple possible worlds. Lisp and other languages can 
fulfil these needs. 
One solution is to build a hybrid system. An 
inference engine was built which uses a feature- 
structure language fo r representation and pattern- 
matching tasks, while using an ATMS to per- 
form conjunctive implications, represent assump- 
tions, represent possible worlds, and maintain the 
truth of derived belief networks when nonmonotonic 
changes occur. The ATMS allows the system to rep- 
resent, and reason with, all consistent possibilities 
at the same time-not just the current best choice. 
In particular, this permits multiple possible inferred 
plans to be output, and multiple possible observa- 
tions to be input. This capability will become im- 
portant for possible utterance disambiguation. 
The system interprets the results of the ATMS 
by using a five-valued uncertainty logic consisting of 
the uncertain belief values ACTUAL, POSSIBLE, HY- 
POTHETICAL, INCONSISTENT, or NULL. Each asser- 
1 Currently, most dialog understanding systems start with 
the assumptions that the hearer and speaker always under- 
stand each other perfectly, that they automatically want to 
cooperate as much as possible, and that they have absolutely 
no other commitments outside of the conversation. Clearly 
some of these assumptions can occasionally be incorrect. 
428 \] 
\[\[action \[\[RELN tthort-antsuor-nogativo-sot-1\] ....... 
\[AGEN ?queat loner'\] \] \]. O--,~,c:{~ i 
\[procl \[ \[IILEI, H S-REQUEST\] ~ :':~" 
t::i ........ i \[OBJE \[ \[REI.~ INFORMI F\] \[AGE// ?answerer\] , ,:>-a~ \[0BJE \[ \[RELN ?vorb\] 
?ro~tl\]\]\]\]\]\] --r- -" i \[docl \[\[REL~ Iio-~EGATIVE\] / \[~)-~ 
\[AGE~ ?ans~oror\].l\] 
\[doc2 \[ \[RELH HEG ATE'\] 
\[OBJE \[ \[RELH ToIru-STATI VE\] 
\[AGEH ~XO1 \[3\] 
\[OBJE I\] \[ RI';IA~ ?vorb\] 
\[~:~ +x0~\] 
\[nBJI.: \[\]\]\]\]\]\]\]\] ;(~u;ll) 
\[affl \[ \[RELH PDSS IBLE-ERPECT\] 
\[OBJE \[ \[RELg Ta-PERFECTI VE\] 
\[OBJE \[ \[IL~3..~ Wakara-1\]\]\]\]\]\]\] \] 
Figure 1 
tiou is assigned a vahw. Not(: that merely because an 
assc'rtion ix l>elieved (.rue in all l,:nc)wn consisl.('nt l)OS - 
nil>h' worlds, this (lo<'s \]Jot mean that it is ACTL!AI, it 
might only bc VOSS~))LE. So<: \[Myc89a\] for (le(ails. 
Plan Schemata. The plan schemata are best 
explained by an actual cxamphx :~' See Figure 1. Plan 
,~'ch('mata a.re lbrmed from possil)ly cyclic fi~al,ure 
strucl, ures and can incl.de varial>l,'s, co-iustanc(: 
variahh's aud rest varial)\[o,c A sch(.ma has a 1t;1111(% 
a s('ri('s of l>r(wolldili<)ns, a series o(" (hx'omt>ositions, 
au<l a series of (~(l'ect.s. 
Plal~ Inf(q'ence. l)la)) infi'r<'m'e is l>e)'formc.d 
hy iml>h,m<')~(ing too<Ms for pla)) rec<>gnition, pre- 
diction, and iufi,rcnce on i.oI> <)film inferenc(" engine 
(thus, it, also use~; fi'ature strm'tures as its maiu dai,a 
rcprescni,a(,ion). I{ecognii km ix based o. 'rcq.in:d on- 
/ailment and use, a slrong mo(M i.\]mI, slates (rll~lt, 
the conju)wtion of f.h(' (h'coml><)si(.i<ms plus l>)'(~con(\]i - 
fi<)ns iml>li('s r('cogni(.ion ,>f the ;wl.i<m (in a l>(>l.(onl: 
up fashion). If a w('alwr )*eco~;)fi(.io)) m('(.hod is de= 
sired, (lu' ust, r cat) Sl>ecil'y ,vu.\[lictcnc!j s<'ls (>f parl,ic-- 
!liar l>r,'c<>,di(.ions aml d('COml><)siti<)))s t<> recog))ize 
Ih<' a<'l.i(>n. (l';.g., {t)re<:l, <1,'c2} is slllli<;i(')fl, for lh(' 
previous exa)nple.) ;~ If the enl.ailme))t is not )'cqutrrd, 
an assu))q>tion I,l.(.i the anl.e<'ed('nl.s d<)in fact im- 
ply the re<'ognized ac(.i<))) can l>(, hlc<)rpor;l.(.(><l in the 
con.j,)uc(.iou, .yi(,hling a c,>),liti(>,,~l r,'<x>gnil.i<m. 
The model fin" I)lal~ I)rcdi<'ti<m re<l.ircs {,hat each 
ass,,rl.io)~ l>e (luI>lica(.('<l in :t i>aralM l.ol)-d<>w,~ n('|.= 
wo)'k W}lOr(' il. is l)larl,:etl ;IS PILH)ICTI';D. 11\] i,\]liS 
case, a I>r('.dicld axtioJ~ infl>li('s ea<'h of the pr('con- 
<lil.ious ~I, II(\] dcco),q>,):d(i<))is, and a), ('fl'c<'( iml)ti<'s a 
l>r<'dict('<l at\[ion. I)lal~ inf(~renc<. <'Olnl)ris<'s a )~m(,<'}) 
I~('(.v¢('~m r,~<:ogniz~d and pr('dic(,~xl ass(,rl, ions. 
'2This is an (al,fidged) plan ~,chcm;~ I() )'cc()gnize a short 
~),I|SW(~I" ill\[Af!l'.\[I-C(il)ll N(~{, ill .\]~tl);t,ll(',S~'t , (!.g. "Aiill;Li-~ho W~t ()- 
,,,,,-hl ,les,, k;(?""lie. M,,(lei,,,as,',,."("(I),,) (you) I,;,ve Ihc 
~,,,,,,,.,,:,..,cnt'e""N.,. (1) ,I,,.'(h,,v,,,"\[~i,\]). Sh,,,.t a.~w,.... 
are formed in Jalwtnese by rCl)e;tting |he vm'l). AII,h(mgh these 
are semantically and I)ragmalically wvll f.rmcd, (hey ,:am.n, 
I)c tran..,latcd liLcrally Iml must Iw rccogniv, ed and lrcumferred, 
sin.'c I';nglish f..vms sh,,r( ;mswcr.', by repealing Lhc auxiliary 
("No. I dos'l."). The kqv fcalmc is (h;~.l the w,b is rclw;dcd 
wit.h(>,~l, an ()l>je(:(., Ml.cr an i,)f(>rm-if (yes/no) qucstio,,. 
aThis is ~t re|il)('nl,ql(. ()f l'{)i,)h\]o<'k's ne(:css;uy ;u.I :;ulIi,:ien( 
(:,,,,,li(.i,,)m \[I<,.)SS\]. 
I 
{ #Kt, ctl.vat~-u,lll) \[l~lit~ tl~tttl 
rREO~,CBS\[$ {l~/ll IFI£1 5U|t~I tl/Wl,~\[\] 
(00 icl ~1~1 \[~p101olI( ~(~ul\]} 
N oE 
ii 
/,LJ 
• o~, c~{ ¢.: 
O a(') 
@IO 
uli~f" 7_ > / ) l l/7k * 
...... 
(~ \[oI1.~ \[)~et)l at0).er~.u~ tin) H,'a*l 1 
( ~ \[ (EELH ~HI \[~gr~ll -- \[ea~cl r@a~) 
IRI/I~ I ,I:~yLIhl \] \] .,\[U/~ L~ .) 
( ll)IEIII i ~ i \[ii \[,O~OKI o~UIII \] 
(Pill\[ DI C/tl \[ \[~iUI \] \] 
I~' lro-c 12 
i} ....... 
Figaro 2 
Since the. hlferencc engine is ill turn based on an 
ATMS, the plan infereuce n'ethod also inherits (he 
capabilities of representing po,;sibh. (assulned) or ac- 
tual asserlions. The representation carefully distin- 
guishes between actual, possible, predicted, and hy- 
pol.hctical occurrences. M ul\[iple self-consistent pos- 
sibilitics can be represented. This results in inforred 
plans also being possible or actual. 
Norm'ally, i( is i>ossible to implemen\[ other too<t- 
el.'+ <,f recognilion, I>rediction, or inference, .sing the 
s)slx'm 'Fh<~ formal philosophical foum\]ali<>us of 
I>\]an inference are not w,'\]l und<.rstood 0".g, dir(.<q, 
imlire<'t, a)M int<'ra<'ting causes \[Pea88!; or (rigg,'r-- 
ing vs. suppor(ing <'ousts), aml are being researchd, 
A <'a,'eful examination of the model sl>ocificatians 
shown hcrc reveals that the current systen~ infers 
plans having monotonic actions (although particular 
st.ares may be. retracted in a nomllonotonic fashion). 
N(mmonotonic extensions are being inw'stigated. 
Execution. There are three kinds of input to 
the systt'm. First, the plan <le,,dgn<:r specifies a se- 
ries of plan s<:henlata. Next, workl knowh'dgc as- 
sc)'ticms and hyl>oMwtical utt,c:r;mces, in the form (ff 
f<'at.re stru<:t.res, a, re f<'d in(:o Lhc sy~t<.)J~ and l>r< • 
ins(an(iatd, l,ater, possiMe (can<lidatc) or actual 
,tterances are i;~put into tim system and used for 
I>l;t,, inDrerwing= 
'l'he resull.s <:onsisL of a n<:twork of iufer<mcas 
aml recognized plans, representc<t insi<h: the ATMS. 
'l'his structured kn<>wle<tlre base can bc <luerie<l by 
the transfer or general, ion systems to provi<le iu-- 
formatiol, 1,o resolve \]>rol>h'ms. AIterl,atively, side- 
effe<:ting <h,m<ms attache<l to sele<'ted ATMS nodes 
can process and report plan infiq'cnce information 
auton<)t,l<),,sly, l)enlons are also used to set a.nd 
deleie processing flags in the ATMS network (e.g., 
for printing out the results of an inferred l>lan only 
olwe, ewm though t, he results are conl, immlly t, ru<'). 
Techique. Inl)ul, plan schemata are fed to tlw 
I)lan--schema interpreter, which breaks i, hen~ Ul> and 
creat<~s an irish'action rule fo.r each \[>re<'on<lition or 
do<'Oml>osi(ion in (.h(' s<'lwma. An last.ruction rule 
c<))Isis(.s of a l>att.er)l (with va.riables) l.<) l><' r(x'(~g 
I)iz('d as an aul.eced('nt, l>lus insl.ru<'l.ions (wilh vm'i 
'2 429 
ables) as the consequent. The instruction rules are 
fed to the nonmonotonic rewriting system \[Has89\] 
and used as rewriting rules. 
Preinstantiation assertions are input to the 
rewriting system. One assertion may match sev- 
eral instruction rule patterns. The recognized con- 
sequents consist of instructions with instantiated 
variables. These are fed to the instruction inter- 
preter, which follows the instructions and instan- 
tiates hypothetical nodes and implications in the 
ATMS corresponding to instantiated components of 
the plan schema. The resulting action and effect as- 
sertions are fed back to the rewriting system as more 
hypothetical input, to instantiate whole networks 
bottom-up from single input facts. Since the ATMS 
uses "uniquification" and never reinstantiates exist- 
ing assertions, this process eventually terminates. 
Run-time utterances are submitted to a feature- 
structure hash test which checks to see whether they 
have identically been instantiated in the ATMS be- 
fore (e.g., hypothetically, etc.). If they have, no 
further pattern matching is required, and the corre- 
sponding node is asserted. Otherwise, the input ut- 
terance is submitted to the rewriting system for pat- 
tern matching and hypothetical instantiation propa- 
gation, as before. Following this, the corresponding 
node is asserted as possible or actual. Assertion trig- 
gers a fast spreading activation in the truth mainte- 
nance network (using bitvectors) which maintains all 
consistent "possible worlds". A demon attached to 
a particular node fires when that node first becomes 
possible or actual. Demons can reset themselves by 
deleting support node flags. 
Current Status. NP version 2.0, reported here, 
has been finished and demonstrated. Currently 
instantiation is done in a bottom-up fashion. A 
graphic output program allows display of the ATMS 
network. Nodes can be moused for input or exami- 
nation. A browsing editor needs to be improved to 
allow relevant parts of the network to be examined. 
The system currently works stand-alone and is ready 
to be integrated with a parser, a transfer system, and 
a language generator when they have been finished. 
Applications. The plan inference system un- 
derstands ongoing task-oriented conversations be- 
tween two people, on the subject of registering for a 
conference. The system serially processes the utter- 
ances, maintaining a representation of the currently 
believed concepts as the conversation progresses. 
Currently, input to the system is a corpus of five 
conversations (20 utterances each, on average) repre- 
senting the expected feature-structure output of the 
ATR parsing system, as generated by the parsing 
researchers. To date, 53 plan schemata have been 
written, dealing with conversation opening and clos- 
ing sequences, "inform-if" (yes/no) questions and 
answers, short answers to questions, ability utter- 
ances, inferred knowledge, wants and intentions, do- 
main plans, idioms, and common-sense knowledge. 
The system understands portions of all five conversa- 
tions. Research is ongoing in this area. In addition, a 
separate set of 46 plans duplicating the "four-layer" 
recognition model \[AI89\] in feature structures has 
been implemented and used to understand an en- 
tire surface-speech-act-level input conversation with 
20 speech-acts (see Figure 2); 232 feature-structure 
assertions were instantiated. 
Future Work. A manual is being written. The 
instantiation method must be improved, l~epresent- 
ing multiple possible nonmonotonic plans is being 
researched. It will also be necessary to represent 
and infer plans containing conditional branches, for 
information-gathering plans in the sample dialogs 4. 
However, the main fimlre research deals with the de- 
sign of a disambiguation system. Disambiguation is 
an evidential reasoning problem, and will probably 
require that a causal reasoning system be built and 
integrated with the multiple-world ATMS. 
Naturally, parallel research in illocutionary and 
perlocutionary force is also continuing. One of the 
first things that must be done is to build a module 
that uses possible plans to resolve zero pronouns. 
Lack of a fully resolved utterance is hindering plan 
recognition. Other work involves representing and 
reasoning with intentions found in a dialog. 
Coneluslon. A working plan inference system 
that uses feature structures has been d,.scribed. The 
system can represent and reasou with assumptions, 
multiple possible inputs, and multiple possible re- 
sults. Preinstantiation of hypothetical assertions al- 
lows inference by fast spreading activation. The re- 
sulting system can be used to infer plans directly 
from the output of a s¢.naantic~, parser, and provide 
information directly to a transfer or generation sys- 
tem, as part of an automatic interpretation system. 
4E.g., "Do you have the announcement?" "No." "Then I will 
give you the details:..." 
430 3 

References 

\[AI89\] Hidekazu Arita and Hitoshi Iida. Tri-Layered Plan 
Recognition Model for Dialogue M~achine Transla- 
tion. Technical Report TR-1-0067, ATR, Kyoto, 
Japan, 1989. (in Japanese). 

\[Den87\] Daniel C. Dennett. The Intentional Stance. The 
MIT Press, Cambridge, Mass. , 1987. 

\[Ilas89\] Toshiro Hasegawa. The Feature Structure Rewrit- 
ing System Manual. TR-1-0093, ATR, Kyoto, 
Japan, 1989. (in dapanese). 

\[KA86\] Hellry A. Kautz and James F. Allen. Generafized 
plan recognition. In 5th AAAI'66, pages 32-37, 
Philadelptfia, Ph., 19S6. 

\[Kno88\] Craig A. Knoblock. Data-driven plan recognition. 
March 1988. CS Dept., CMU, Pittsburgh, PA 

\[Mye88\] John K. Myers. The necessity of intentions under 
fallible execution. 1988. Unpublished manuscript. 

\[Mye891\] John K. Myers. An assttmption-based plan infer- 
ence system for conversation understanding. In 
WGNL Meeting o\] the IPSJ, pages 73-80, Old- 
nawa, Japan, June 1989. 

\[Mye89b\] John K. Myers. The ATMS Manual (Version 1.1). 
Tech. Rep. TR-1-0074, ATR, Kyoto, Japan, 1989. 

\[Pea88\] Judea Pearl. Probabilistie Reasoning in Intelligent 
Syotems. Morgan Kaufmann Publishers, Inc., Los 
Altos, CA., 1988. 
