TOWARDS A SELF-EXTENDING PARSER 
Jaime G. Carbonell 
Department Of Computer Science 
Carnegie-Mellon University 
Pittsburgh, PA 15213 
Abstract 
This paper discusses an approach to incremental 
learning in natural language processing. The 
technique of projecting and integrating semantic 
constraints to learn word definitions is analyzed 
as Implemented in the POLITICS system. 
Extensions and improvements of this technique 
are developed. The problem of generalizing 
existing word meanings and understanding 
metaphorical uses of words Is addressed In terms 
of semantic constraint Integration. 
1. Introduction 
Natural language analysis, like most other subfields of 
Artificial Intelligence and Computational Linguistics, suffers 
from the fact that computer systems are unable to 
automatically better themselves. Automated learning ia 
considered a very difficult problem, especially when applied 
to natural language understanding. Consequently, little effort 
ha8 been focused on this problem. Some pioneering work in 
Artificial intelligence, such as AM \[I\] and Winston's learning 
system 1"2\] strove to learn or discover concept descriptions 
in well-defined domains. Although their efforts produced 
interesting Ideas and techniques, these techniques do not 
fully extend to • domain as complex as natural language 
analysis. 
Rather than attempting the formidable task of creating a 
language learning system, I will discuss techniques for 
Incrementally Increasing the abilities of a flexible language 
analyzer. There are many tasks that can be considered 
"Incremental language learning". Initially the learning domain 
Is restricted to learning the meaning of new words and 
generalizing existing word definitions. There ere a number of 
A.I. techniques, and combinations of these techniques 
capable of exhibiting incremental learning behavior. I first 
discuss FOULUP and POLITICS, two programs that exhibit a 
limited capability for Incremental word learning. Secondly, the 
technique of semantic constraint projection end Integration, 
as Implemented in POLITICS, Is analyzed in some detail. 
Finally, I discuss the application of some general learning 
techniques to the problem of generalizing word definitions 
end understanding metaphors. 
2. Learning From Script Expectations 
Learning word definitions In semantically-rich contexts Is 
perhaps one of the simpler tasks of incremental learning. 
Initially I confine my discussion to situations where the 
meaning of a word can be learned from the Immediately 
surrounding context. Later I relax this criterion to see how 
global context and multiple examples can help to learn the 
meaning of unknown words. 
The FOULUP program \[3\] learned the meaning of some 
unknown words in the context of applying s script to 
understand a story. Scripts \[4, 5\] are frame-like knowledge 
representations abstracting the important features and 
causal structure of mundane events. Scripts have general 
expectations of the actions and objects that will be 
encountered in processing a story. For Instance, the 
restaurant script expects to see menus, waitresses, and 
customers ordering and eating food (at different 
pre-specifled times In the story). 
FOULUP took advantage of these script expectations to 
conclude that Items referenced in the story, which were part 
of expected actions, were Indeed names of objects that the 
script expected to see. These expectations were used to 
form definitions of new words. For instance, FOULUP induced 
the meaning of "Rabbit" in, "A Rabbit veered off the road 
and struck a tree," to be a self-propelled vehicle. The 
system used information about the automobile accident script 
to match the unknown word with the script-role "VEHICLE", 
because the script knows that the only objects that veer off 
roads to smash Into road-side obstructions ere self propelled 
vehicles. 
3. Constraint Projection In POLITICS 
The POLITICS system E6, 7\] induces the meanings of 
unknown words by a one*pass syntactic and semantic 
constraint projection followed by conceptual enrichment from 
planning and world-knowledge inferences. Consider how 
POLITICS proceeds when It encounters the unknown word 
"MPLA" In analyzing the sentence: 
"Russia sent massive arms shipments to the MPLA In Angola." 
Since "MPLA" follows the article '*the N it must be a noun, 
adjective or adverb. After the word "MPLA", the preposition 
"in" Is encountered, thus terminating the current 
prepositional phrase begun with "to". Hence, since all 
well-formed prepositional phrases require a head noun, and 
the "to" phrase has no other noun, "MPLA" must be the head 
noun. Thus, by projecting the syntactic constraints 
necessary for the sentence to be well formed, one learn8 
the syntactic category of an unknown word. it Is not always 
possible to narrow the categorization of a word to a single 
syntactic category from one example. In such cases, I 
propose Intersecting the sets of possible syntactic 
categories from more then one sample use of the unknown 
word until the Intersection has a single element. 
POLITICS learns the meaning of the unknown word by a 
similar, but substantially more complex, application of the 
same principle of projecting constraints from other parts of 
the sentence and subsequently Integrating these constraints 
to oonetruot a meaning representation. In the example 
above, POLITICS analyzes the verb "to send" as either in 
ATRANS or s PTRAflS. (Schank \[8\] discusses the Conceptual 
Dependency case frames. Briefly, a PTRANS IS s physical 
transfer of location, and an ATRANS Is an abstract transfer 
of ownership, possession or control.) The reason why 
POLITICS cannot decide on the type of TRANSfer is that it 
does not know whether the destination of the transfer (i.e., 
the MPLA) Is s location or an agent. Physical objects, such 
as weapons, are PTRANSed to locations but ATRANSed to 
agents. The conceptual analysis of the sentence, with MPLA 
as yet unresolved, Is diagrammed below: 
*SUSSIA* <-~ 
 \[CIPSl <is> LOC vii ~qNGOLAe 
t 
l mlq.R) 
RTRRNS • d IN, iq\[CIPill 
I IN< ,,ffi/$SIRi, 
I J~ERPONe <ls~ NWISER vii (, llOMI) 
What has the analyzer learned about "MPLA" as s result of 
formulating the CD case frame? Clearly the MPLA can only be 
an actor (I.e., s person, an Institution or s political entity in 
the POLITICS domain) or s location. Anything else would 
violate the constraints for the recipient case In both ATRANS 
end PTRANS. Furthermore, the analyzer knows that the 
location of the MPLA Is Inside Angola. This Item of Information 
is integrated with the case constraints to form a partial 
definition of "MPLA". Unfortunately both Iocatlcms and actors 
can be located inside countries; thus, the identity of the 
MPLA is still not uniquely resolved. POLITICS assigns the 
name RECIP01 to the partial definition of "MPLA" and 
proceeds to apply Its Inference rules tO understand the 
political Implications of the event. Here I discuss only the 
Inferences relevant for further specifying the meaning of 
-MPLA m . 
4. Uncertain Inference in Learning 
POLITICS Is a goal-driven tnferencer. It must explain ell 
actions In terms of the goals of the actors and recipients. 
The emphasis on inducing the goals of actors and relating 
their actions to means of achieving these goals is Integral to 
the theory of subjective understanding embodied in 
POLITICS. (See \[7\] for a detailed discussion.) Thus, POLITICS 
tries to determine how the action of sending weapons can be 
related to the goals of the Soviet Union or any other possible 
actors involved in the situation. POLITICS k~s that Angola 
was Jn a state of civto war; that Is, a state where political 
factions were .'xerclstng their goals of taking military and, 
therefore, political control of a country. Since po6ssssing 
weapons Is a precondition to military actions, POLITICS infers 
that the recipient of the weapons may have been one of the 
poliUcal factions. (Weapons ere s means to fulfUllng the goal 
of • political faction, therefore POLITICS Is able to explain 
why the faction wants to receive weapons.) Thus, MPLA Is 
Inferred to be a political faction. This Inference is Integrated 
with the existing partial definition and found to be 
consistent. Finally, the original action Is refined to be an 
ATRANS, as transfer of possession of the weapons (not 
merely their k:mation) helps the political faction to achieve 
Its military goal. 
Next, POLITICS tries to determine how sending weapons to s 
military faction can further the goals of the Soviet Union. 
Communist countries have the goal of spreading their ' 
Ideology. POLITICS concludes that this goal can be fulfilled 
only if the government of Angola becomes communist. Military 
aid to s political faction has the standard goal of military 
takeover of the government. Putting these two facts 
together, POLITICS concludes that the Russian goal can be 
fulfilled if the MPLA, which may become the new Angeles 
government, is Communist. The definition formed for MPLA Is 
ae follows: 
QI~'I i~a1"~ tntrvI (OPS flPLA (POS NOUN (TYPE PROgI\[R))) 
(TOK efllq.A.) ) 
(PARTOF. luRN6OLR.) (|oEOLOGY . ~¢OiltlUN|STe) 
(GORLSt ((ACTOR (*flPLA*) iS (SCONT O§JI\[CT (dN6OLRe) 
Vm. (IR)))))P 
The reason why memory entries are distinct from dictionary 
definitions is that there is no one-to-one mapping between 
the two. For Instance, "Russia" and "Soviet Union" are two 
separate dictionary entries that refer to the same concept in 
memory. Similarly, the concept of SCONT (social or political 
control) abstracts Information useful for the goal-driven 
inferences, but has no corresponding entry in the lexicon, as 
I found no example where such concept was explicitly 
mentioned In newspaper headlines of political conflicts (i.e., 
POLITICS' domain). 
Some of the Inferences that POLITICS made are much more 
prone to error than others. More specifically, the syntactic 
constraint projections and the CD case-frame projections 
ere quite certain, but the goal-driven Inferences are only 
reasonable guesses. For Instance, the MPLA coWd have been 
• plateau where Russia dePosited Its weapons for later 
delivery. 
5. A Strategy for Dealing with Uncertainty 
Given such possibilities for error, two possible strategies to 
deei with the problem of uncertain inference come to mind. 
First, the system could be restricted to making only the more 
certain constraint projection and integration inferences. This 
does not usually produce s complete definition, but the 
process may be Iterated for other exemplars where the 
unknown word Is used in different semantic contexts. Each 
time the new word Is encountered, the semantic constraints 
are integrated with the previous partial definition until a 
complete definition is formulated. The problem with this 
process Is that it may require a substantial number of 
iterations to converge upon s meaning representation, end 
when it eventually does, this representation wtll not be as 
rich as the representation resulting from the less certain 
goal-driven inferences. For Instance, it would be impossible 
to conclude that the MPLA was Communist and wanted to 
take over Angola only by projecting semantic constraints. 
The second method is based on the system's ability to 
recover from inaccurate inferences. This is the method i 
implemented in POLITICS. The first step requires the 
deteotlon of contradictions between the Inferred Information 
end new Incoming information. The next step is to assign 
blame to the appropriate culprit, i.e., the inference rule that 
asserted the incorrect conclusion. Subsequently, the system 
must delete the inaccurate assertion and later inferences 
that depended upon it. (See \[9\] for a model of truth 
maintenance.) The final step is to use the new information to 
correct the memory entry. The optimal system within my 
paradigm would use a combination of both strategies - It 
would use Its maximal Inference capability, recover when 
Inconsistencies arise, and iterate over many exemplars to 
refine and confirm the meaning of the new word. The first 
two criteria are present in the POLITICS implementation, but 
the system sto~s building a new definition after processing a 
single exemplar unless it detects a contradiction. 
Let us briefly trace through an example where PC~.ITICS la 
told that the MPLA is indeed a pisteau after it inferred the 
meaning to be a political faction. 
I POLITICS Pun -- 2/06/76 ! 
• : INTERPRET US-CONSERVRT IVE) 
INPUT STORY, Russia sent massive arms ship.eats 
to the flPL.A in Re,gels. 
PARSING... (UNKNOUN UOROI MPLA) 
:SYNTACTIC EXPECTATION! NOUN) 
(SERRNTIC EXPECTATION; (FRANC: (ATRONS PTRONS) SLOTI RECIP 
REQ, ILOC ROTOR))) COflPLETEO. 
CREATING N( u MEMORY ENTRY, *flPLRo 
INFERENCE, ~,MPLRo MIAY BE A POLXTICI:n. FACTION OF mARGOt.fiG 
|NFEfl(NCE, eflUSSIAe RTRRNS eRRMSo TO tAPLRo 
INFERENCE; *MPLAe IS PNOOROLY aCOflMUNXSTe 
INFERENCE, GOAL OF aMPLRa IS TO TAK( OVEN eANOOl.Ae 
INSTANTIATING SCAIPTJ SRIONF INFERENCE; 
GOAL OF eRUSSIAa I$ toNGOLflo TO BE ¢comflNl|$Te 
I Question-salem- dialog ) 
441hst does the MPLA ~ent the arms foP? 
TNE RPLR MANTa TO TAKE OVER RNGOLR USING THE NEIMONS. 
I~he( might the ether factionS in An(iolll de? 
THE OTHER FACTIONS NAY ASK SORE OTHER COUNTRY FOR RRflS. 
| Reading furthcP Input \] 
INPUT STORY; +The Zunqabl faction oleoPatlng fPoe the I~PLA 
plateau received the $ovist uealNme. 
PARS |NO... CONPLETEO • 
GREAT|NO NEW N(NORY ENTRY: aZUNGRO|a 
ACTIVE CONTEXT RPPLJCRItLE, ~IONF 
C1 ISR CONFLICT, eMPLRe ISR (eFRCTIONo sPI.RTERUe) 
(ACTIVATE' (|NFCN(CK C|)) R(OUEST(O 
C2 SCRIPT ROLE CONFLICT, 
(&R\[O-RECXP |N SRIOMF) • aMPLRe RNO aZUNGABIe 
(ACTIVATE (INFCHECK C2)) RE~JEST\[O 
(INFCHECK C1 C2) INVOKEOt 
RTTERPT TO MERGE MEMORY ENTRIES, (*M~.Ae aZON~Ia)...FAIUJRE' 
INFER(lICE RULE CHECK(O (RULEJFI . SRIOMF)...OK 
INFERENCE RUt.E CHECKED (flULEIGO)...CONFLICT! 
OELETING RESULT OF RULE/GO 
C2 RESOt.VEDt ~f'~'LRe \]SA *PLRTEIqJe IN eRNGOLRs 
C2 flESOLVEO; UlAI?-RECIP IN SRIOMF) • eZONGROIo 
REDEFINING enPLRe AS eZUNGRe|O...COMPI.IrTEO. 
CREATING HEM orlPLRo fl(NORY (NTNY...CORPLET(O. 
POLITICS realizes that there is an Inconsistency In Its 
Interpretation when It tries to integrate "the MPLA plateau" 
with its previous definition of "MPLA". Political factions and 
plateaus ere different conceptual classes. Furthermore, the 
new Input states that the Zungsbl received the weapons, 
not the MPLA. Assuming that the Input Its correct, POLITICS 
searches for an Inference rule to assign blame for the 
present contradiction. This Is done simply by temporarily 
deleting the result of each inference rule that was activated 
in the original interpretation until the contradiction no longer 
exists. The rule that concluded that the MPLA was a political 
faction Is found to resolve both contradictions If deleted. 
Since recipients of military aid must be political entitles, the 
MPLA being s geographical location no longer qualifies as a 
military aid recipient. 
Finally, POLITICS must check whether the inference rules 
that depended upon the result of the deleted rule are no 
longer applicable. Rules, such as the one that concluded that 
the political faction was communist, depended upon there 
being a political faction receiving military aid from Russia. 
The Zungabi now fulfll:s this role; therefore, the inferences 
about the MPLA are transfered to the Zungabl, and th~ MPLA 
Is redefined to be a plateau. (Note: the word "Zungabl" was 
constructed for this example. The MPLA is the present ruling 
body of Angola.) 
6. Extending the Project and Integrate Method 
The POL)TICS Implementation of the project-and-integrate 
technique ts by no means complete. POLITICS can only 
Induce the meaning of concrete or proper nouns when there 
Is sufficient contextual information In a single exemplar. 
Furthermore, POLITICS assumes that each unknown word will 
have only one meaning. In general It is useful to realize when 
a word Is used to mean something other than Its definition, 
and subsequently formulate an alternative definition. 
I Illustrate the case where many examples are required to 
narrow down the meaning of s word with the following 
example: "Johnny told Mary that If she didn't give him the 
toy, he would <unknown-word) her." One can induce that the 
unknown word Is a verb, but its meaning can only be guessed 
at, In general terms, to be something unfavorable to Mary. 
For Instance, the unknown word could mean "take the object 
from", or "cause injury to". One needs more then one 
example of the unknown word used to mean the same thing 
In different contexts. Then one has s much richer, combined 
context from which the meaning can be projected with 
greater precision. 
Figure 1 diagrams the general project-and-integrate 
algorithm. This extended version of POLITICS' word-learning 
technique addresses the problems of iterating over many 
examples, multiple word definitions, and does not restrict its 
Input to certain classes of nouns. 
7. Generalizing Word Definitions. 
Words can have many senses, some more n"neral than 
others. Let us look at the problem of gen lizlng the 
semantic definition of a word. Consider the case where 
"barrier" is defined to be a physical object that dlsenables a 
transfer of location. (e.g. "The barrier on the road Is blocking 
my way.") Now, let us interpret the sentence, "Import quotas 
form a barrier to International trade." Clearly, an Import quota 
Is not • physical object. Thus, one can minimally generalize 
"barrier" to mean "anything that disc.shies s physical 
transfer of location." 
Let us substitute "tariff" for "quota" In our example. This 
suggests that our meaning for "barrier" is insufficiently 
general. A tariff cannot disensble physical transfer; tariffs 
dime.able willingness to buy or sell goods. Thus, one can 
further generalize the meaning of barrier to be: "anything 
that dlaenablee any type of transfer", Yet, Urea trace of the 
FIght 1: The prijeat-a.d-lntsgPete Nthed far Indu@l~ Re. ueP4 and :oe~ept detlnitleml 
contalnl.| •hi URK~O~ •lard 
PROJECT the s~ntaetie Imd 
semantic ¢onstrai.tl! fPoa eft Imelvslt of 
the other eowDonints 
\]N~qRTE • 1! Oh• ©onttrilntl 
tQ tM, imlite • wd deflflltl(m 
INTEGRRTE 91ob•l Cento•t to 
(mrlch 4Qtlnitiqm 
I COn•cut"air OlealmPseaedelp 
Jml goil,.dPiwm Int.fqm~te 
NO 
emcm~ in t M, Imee~. u•Ing a I•eet- 
q:m" •also-! IP•| 
NO \[111101 
Postul•te • mm .erd same aml 
build a I terlqlte defiflitie~ 
Delete culpell Inf•r~e mid ~.J 
generalization process must be remembered because the 
original meaning is often preferred, or metaphorically 
referenced. Consider: "The trade barriers were lifted. • and 
"The new legislation bulldozed existing trade barriers. • 
rheas sentences can only be understood metaphorically. 
rhat is, one needs to refer to the original meaning of 
~barrier" as a physical object, In order for •lifting" or 
'bulldozing" to make sense. After understanding the literal 
leaning of a "bulldozed barrier", the next step Is to infer 
he consequence of such aft action, namely, the barrier no 
)nger exists. Finally, one can refer to the generalized 
leaning of "barrier" to interpret the proPoaltion that •The 
ew legislation caused the trade barriers to be no longer In 
xietence." 
propose the *ollowing rules to generalize word definitions 
ld understand metaphorical references to their ortglnol, 
mmel definition: 
1 ) If the definition of a word violates the semantic 
constraints projected from an interpretation of the 
rest of the sentence, create a new word-sense 
definition that copies the old deflnltiml minimally 
relaxing (I.e., generalizing) the violated constraint. 
2) In Interpreting new sentences always prefer 
the mast specific definition if applicable. 
3) If the generalized definition Is encountered 
again in Interpreting text, make It part of the 
permanent dictionary. 
4) If • word definition requires further 
generalization, choose the existing most general 
definition and minimally relax Its violated semantic 
constraints until a new, yet more general definition 
Is formed. 
5) If the case frame formulated in interpreting a 
sentence projects more specific semantic 
constraints onto the word meaning than those 
consistent with rite entire sentence, Interpret the 
word usln(! the most specific definition conslste.t 
with the case frame. If the resultant meaning of 
the case frame Is inconsistent with the 
interpretation of the whole sentence, Infer the 
most likely consequence of the pMtlally-build 
Conceptual Dependency case frame, and use this 
consequence In Interpreting the rest of the 
sentence. 
The process described by rule 5 enables one to Interpret the 
metaphorical uses of words like "lifted" and "bulldozed" In 
our earlier examples. The literal meaning of each word i8 
applied to the object case, (i.e., "barrier•), and the Inferred 
consequence (i.e., destruction of the barrier) i8 used to 
Interpret the full sentence. 
8. Coral.cling Remarks 
There are a multitude of ways to incrementally Improve the 
language understanding capabilities of a system. In this 
paper I discussed in some detail the process of learning new 
w~rde. In lesser detail I presented some ideas on how to 
generalize word meanings and Interpret metaphorical uses of 
individual words. There are many more aspects to learning 
language and understanding metaphors that I have not 
touched upon, For Instance, many metaphors transcend 
Individual words and phrases. Their Interpretation may 
require detailed cultural knowledge \[10\]. 
In order to place some perspective on project-and-integrate 
learning method, consider throe general learning mechanisms 
capable of implementing different aspects of Incremental 
language learning. 
Learning hy example. This Is perhaps the most 
general learning strategy. From several exemplars, 
one can intersect the common concept by, If 
necessary, minimally generalizing the meaning of 
the known part of each example until a common 
aubpart Is found by Intersection. This common 
eubpart Is likely to be the meaning of the unknown 
section of each exemplar. 
Learning by near-miss analysis. Winston \[2\] 
takes full advantage of this technique, it may be 
usefully applied to a natural language system that 
can Interactlveiy generate utterances using the 
words it learned, and later be told whether It used 
those words correctly, whether It erred seriously, 
or whether It came close but failed to understand 
a subtle nuance In meaning. 
Learning by contextual expectation. EasanUally 
FOULUP and POLITICS use the method of 
projecting contextual expectations to the 
linguistic element whose meaning Is to be Induced. 
Much more mileage can be gotten from this 
method, especially If one uses strong syntactic 
constraints and expectations from other 
knowledge sources, such as s discourse model, s 
narrative model, knowledge about who is providing 
the information, and why the information Is being 
provided. 
9. References 
T. 
2. 
3. 
4. 
5. 
6. 
7. 
8. 
9. 
TO. 
Lenet, 0. AMz Discovery In Mathematics as 
Heuristic Search. Ph.D. Th., Stanford University, 
1977. 
Winston, P. Learning Structural Descriptions from 
Examples. Ph.D. Th., MIT, 1970. 
Granger, R. FOUL-UPt A Program that Figures Out 
Meanings of Worcls from Context. IJCAI-77, 1977. 
Schank, R. C. and Abelson, R.P. Scripts, Goals, 
Plans and Unclerstancling. Hillside, NJ: Lawrence 
Erlbaum, 1977. 
Cullingford, R. Script Appllcationt Computer 
Uncleratandlng of Newspaper Stories. Ph.D. Th., 
Yale University, 1977. 
Carbonell, J.G. POLITICS: Automated Ideological 
Reasoning. Cognitive Science 2, 1 (1978), 27-51. 
Carbonell, J.G. Subjective Unclerstancllng: 
Computer Mo<lels of Belief Systems.. Ph.D. Th., Yale 
University, 1979. 
Sohsnk, R.C. Conceptual Information Processing. 
Amsterdam: North-Holland, 1975. 
Doyle, J. Truth Malntenanoe Systems for Problem 
Solving. Master Th., M.I.T., 1978. 
Lakoff, G. and Johnson, M. Towards an 
Experimentalist Philosopher: The Case From Literal 
Metaphor. In preparation for publication, 1979. 

