American Journal ~f Co~putational Linguistics 
Microfiohe 18 
SEMANTICALLY A~*ALY~I G 
AN ENGLISH SUBSET 
FOR THE CLOWliS MICROWORLD 
Rob.ext F. Simo'ns 
Gordon Benne tthNovak 
Department of Computer Sciences 
The University of Texas at Austin 
Copyright 1975 
Association for 
Computationad l~inguE&ti cg 
ABSTRACT 
A microworld system is described for displaying visual 
representations of the meaning of a subset of Eng,lish thak.con- 
cerns a clown that can balance objecbs and can participate in 
motion scenarios. Nouns such as "clown", "lighthouse", "water" 
etc. are programs that construct images on a display screen. 
Other nouns such as "top", "edge", "side", etc, are defined as 
fm~tions that return contact p~ints for the pictures. 
Adjectives and ad rerbs provide data on size and angles of sup- 
port. Prepositions and verbs are defined as semantic functions 
that explicate spatial relations among noun ifnages. Generally, 
a verb praduces a process model that encodes a,series oftscenes 
that represent initial, intermediate and final displays of the 
changes the verb describes. 
The system is programmed in UT.LISP fqr CDC equipment and 
uses an IMLAC display system. It] currently occupies 3210K 
words of core and requires less than a second to translate a 
sentence into a picture. Applications,to teaching linguistics 
and languages are suggested. 
CONTENTS 
VI 
VII 
VIII 
......... Abstract 
~cknowledgrnehts ..... 
~ntroduction ...... 
Background ....... 
Pictorial models . ... 
An English subset gramrnat 
Lexicon ........ 
Grammar ........ 
Stemantics of the subset 
Semantics of prepositions 
Verb semantics .... 
Semantics of scenes . . 
Conclwding discussion . 
References ....... 
Appendix': Clowns program 
FIGURES 
1 State verbs ................ 
2 A motion verb ............... 
.... 
S network ......... 
Np network ................. 20 
Dclause network .............. 
................. PPnekwork 
3 Process model for MQVE* 
Additional pictures on fraanes Z 42 
ACKNOWLEDGMENTS- 
This. resear'.l was supported in part by NSF Grant GJ509E. 
I am indebted 4ia Bill Henfieman, Jonathan Slocurn. Michael K. 
Smi-th, Ken Speaker and Bob Amsler for productive discussians 
and help ih designirig and debugging the programs described here 
My thanks to Professor Woodrow Bledsoe for making available the 
IMLAC- display gnd its operating systems. 
,NATU~W LANGUAGE ESEARCH FOR CAI 
Sponsored by 
THE NATIOW SCIENCE FOUNDATION 
Grant GJ 509E 
Privately circulated as: 
TECHNICAL REPORT NL-24 
Department of Computer Sciences 
THE UESIVERSITY OF TEXAS AT 'AUSTIN 
April 1975 
SEMANTICALLY ANALYZING AN ENGLISlI 
SUBSET FOR THE CLOWNS mCROWORLD 
I Int~oduction 
Several examples of semantically based grammars have appeared in the 
literature since 1970. 
The most corttplete of these are wknogradls (1972') 
outline of a systemic grammar for commanding and questioning the robot hand 
in the MIT blocks world, ~eidorn' s (1972). rewrite rules for anal'yzing and 
generating ~nglish descr'iptions and transforming them into GPSS programs, 
and 'the ATN gramdar of questions for the*Lunar Rocks Data Base presented 
by Woods, Kaplan and Nash-Webber (1972). Most other grammars of significan 
size, such as that of the NYU String Analysis Project (Grishman and Sager 
1973) and Werous gramrs developed for mechanical translation are 
largely syntactic in orientation and not easily accessible. Riesbeck also 
presents a semantic grammar m rne form of a sek of LISP programs to cornput 
conceptual dependencies (1975). 
A difficulty with these reports is that th6 systems using the grammars 
are typically+ quite large programs--loOK+--and the interactions between the 
grammar and the rest of the system are frequently quite complicated. The 
reader who wishes to use them as a basis for constructing small natural 
language understanding system may well be at-a loss as how to-begin. He 
may have the impression that a natural language pro~essing~aystem is a vast 
undertaking involving great complexity of programming. 
He will not be co'mpletely incorrect in these impres.sione, byt in fact 
p-ramming a grammar and. aemantic system fot a micrdworld model to under- 
etand a small subaet of English is no lonuer a formidable task. khe 
6 
vocabulary can 'be restricted to one hundred or so words, a minimally 
sufficient syntactic and semantic syatem can be expremed in a few dozen 
rules supported by a dozen or so aemantic functions, and the pragmatics of 
guch microworlds as the STRIPS robot, the blocks world, or the CLOWNS 
world presented here, can be modelled very stmply, The siqplest microworld 
models that comunicete in English require an effort somewhere between a 
two week homewodc exercise and a graduate term project. CLOWNS represents 
about 6 man-months of effort so far. 
But is there any real purpose in studying English communication in 
these trivial microworld situations? If we mbdel language behavior in 
one microworld we remain eeveral orders of magnitude short of understanding 
the genera2 use of the langu~ge in text, or in verbal discourse and equally 
far from the possible g~al of tnstructing computers In English to accomplish 
a general run of tasks. 
I: remain Incurably optimistic. The generalizations about tiny subeets 
of language and b+avior that emerge from microworld models gradually 
accumulate in bur human minda into what may eventually prove sufficient 
understanding fiv the accompliehment of socrally useful tasks. me 
initiation rltual of programing a mini-intelligence is a r)ecessary 
pre-requisite to programming one that is more sophiaticafed. 
In this paper, CLQWNS, a simple microworld model is presented with an 
explicit tutorial intent. A brief grammar is described that accounts for 
much of the embedding logic of English canstructions; a flystem of trans- 
formations of Eng1iah.condtituents to property list representations of 
semantic network structures is followed by their represen,tation in a 
dynamic process model that can be operated to produce successive states 
deecribed by the English. 
The principles used in the system are.a concise 
representation of my gleanings from recent literature dnd of course from 
work of my own and my students. 
11 Background 
In this section only a few of hundreds o'f natural language processing 
papers are suggested as entries to thelliterature. At least a dozen reviews 
of this liteyature are available; halker's is not only among the most recent 
and complete (Walker 1973), but it includes a section that cites the reviews. 
Since 1970, the langu'age processing literature has been rich in 
reports of natural langbage systems that can understand subspts of English 
with respect to various microworlds. In addition to previously mentioned 
work by Woods, Heidorn and Winoppad, there are less frequently cited but 
quite interesting theses by Badre (19.72) That learns to do very simple 
number problems from text, by Scragg (19-75) that answers questions about food 
preparation processes and by Bruce (1972) that presents a logic and a system 
for answering questions about temporal reference. Schank Riesbeck, Goldman 
and Rieger (1975) have publisheda significant series of papers on Semantic 
parsing., inference and generation for an Endish subset concerning fairly 
ordinary human action$. Hendrix, SLocum and Thompson (1973) describe* a 
systexh for under.s-ding and generating English about commercial transactions 
and Mmple movements. 
Hendrift (1975) has also developed a set theoretic 
system of proteos models for representing natural language meanings. TheSp 
models are descended from robot problem solvidg research by Fikes and Nilsson 
8 
(1971) and siki6ssy et. al. (19i3). Harris (1972) provides a tour de force 
that uses problem solving, inference agd learning methode to teach a robot 
facts about its microworld. Hobbs (1974) presenta an hpproach to natural 
language semantics that ig shown to apply to several applications, diagrams- 
to-language,, English and Algol-to-Algol, e tc. 
Much of the most recent work by Abelson (1975) , Charniak (1972 ) ,' 
Schank and Abelaon (1975), .Mineky (1975), Winograd (1975), Bobrow and Norman 
(1975), Collins and Warnock (1975), Rumelhart (1975) has progressed beyond 
the question of grammar and semantic systems to that of such larger units 
of semantic organization as Frameg, Stpry grammars, Plane, Schemes, Drems, 
etc. Although at this writing most of these formulations still fall short 
of computational realization, it is clear that the research task of the 
immediate future is one of formulating and programing structures of 
organizationrthat will successfully model much more complicated microw~rlds 
than those presently achieved. A forthcoming book edited by Collins and 
Bobrow will present many of these ideas. 
LISP is still the language of most frequent choice for these experiments 
and thanks to the prevalence of virtual memories and virtual LISP,)the 
limitation to inrcore implementations has essentially vaniehed. 
Many of 
the programs cited used require from 100 to 300K10 cell8 of smrage. 
l'he system described in subsequent .eections resides in 32K on a CDC syegern, 
although our moat recent additions have caused ue to use e virtual mbry 
version of UTLI$P that was developed by -Wry Tyson. 
9 
111 Pictor-ial Models 
Ignoring early w~rk largely lost in the archives of corporate memos, 
Winograd's language processor is essentialky a first reporting of how to 
map Englpsh sentences into diagrammatic pictures. Apart from potential 
applications, the pictures are of great valve in providing a universally 
understood snecond language to demonstrate the system's interpretation of 
the English input, While we are still struggling in early stages of how 
to compute from English descriptions or instructions, there is much to be 
gained from studying the subset of English that is picturable. Translation 
of English into other more general languages such as predicate calculus, 
LTSP, Ruseian, Basic English, Chinese, etc. can provm the same feedback 
as to the system's interpretation and must suffice for the unpicturable 
set of English. But for teaching purposes, computing pictures from 
language is an excellent instrument. 
We began with the notion that it should be quire easy to construct a 
microwcizld concerning-a clown, a pedestal, and a pole. The resulting 
system cauld draw pittures for such sentences as: 
A clown holding a pole balance6 on his head in a boat. 
A clown on his arm on a pedestal bglances a call clown or his head. 
Figure 1 shows examples of diagsamg produced in responBe 'to these 
sentences. 
Ue $roereesed fhen to smtcnces concerning movement by adding land, 
water, o lighthouse, a dock and aboat. We were then able to draw pictures 
such as Figure 2 to represent the meanings of: 

A cloh on his head eaile a boat from the duck to the lighthouse. 
In the context of graphics, two dimensional line drawings ate attractive 
in their 3implicfty of computation. 
An obj~ct is defined as a LOGO graphzcs 
program that draws it (see Section VI) A scene is a set of ob~ects re- 
lated in terms of contact points, A scene can be described by a set of 
pradicstes 
(BOAT ABOVE WATER) (ATTACH BOAT* WATEqpl) 
(~CK ABOVE WATER) (DOCK LEFTOF WATER) (BOAT RIGHTOF WK) 
(ATTACH DOCYky WATE%) (ATTACH BOATXl+kY DOCSy 
Orientation functions for adjusting starting points and headings of the 
programs that draw the ob~ects are requlred and these imply some trigono- 
rnetrlc functians A LISP package of about 650 llnes has been developed by 
Gordon Bennett m p~ovide the plcture making capablllty 
What 1s rnalnly relevant,to the computation of language meanings 1s 
that a semantlc structure sufficient to transmlt data to the drawing package 
is easlly represented as a property list associated ulth an artlficlal 
pme for the scene For example, A CLOWN ON A-PEDESTAL" results in the 
following stbqture 
(Cl, TOK CLOWN, SUPPORTBY C2, ATTACH(C1 FEET= C2 TOPXY)) 
(€2, TOK PEDESTAL, SUPPORT C1, ATTACH(C2 TOPXY Cl FEETXI)) 
(CLOUN, EXPRCWDAO ,) FEET XI, SIZE 3, STARTPT XY, HEADING A) 
(PEDES~AL. EXPR(LIU.IBDA() 
) TOP XY, SIZE 3. STARTPT XY, BEADING A) 
A larger scene has more objects mare attach relations, and may ~nclude 
addltlona2 relations such as INSIDE, LEFTOF, RTGHTOF, etc In any case 
the scene is, s'eaantlc+lly represented as a set of objects connected by 

13 
relations in a graph (1 e a semantlc network) that can easllj be stored 
as a property list wlth references to other objects with property lists 
l r 
We take "balance" stand' support "hold ' is on" etc. as state 
describing verbs in contrast to those such as "sail", ' ridef , fly' 
"buy etc whlch descrlbe changes of state To model the meaning of 
state verbs requlres only a single diagram to show the state described 
Far change of state verbs a serles of plctures'is required and a process 
model IS used to construct a sequence of state descrlptlons each of vhlck 
can produce a diagram 
IV An Engllsh Subset Grammar 
We take the Woods ATN as a baslc formalism f~r describing a grammar 
computationally This system has been well-described by Woods (19M), lts 
application to Engllsh semantics by Simmons (1973) and a UTLISP version was 
programed by Matousek & Slocum (1972) Whlle generally ignoring theoretical 
lssues In linguistics, we do use such principles as the fact that sentences 
are composed of constituents, that there are syntactx rules definlng 
acceptable sequences of constibuents. and that.underlying the Engllsh statr- 
ment there is an idea that can be expressed in Sume other language by 
transformations on the Engllsh constituents 
The underlying idea can be 
expressed in a formal language such as some version of predicate logic, or 
in a computer data structure 01 in a langr~age of fumtlons and arguments such 
In presenting the.€ollowinp grammar and semantlc system our emphasis 
~s on dealing with the highly dariable nature of Fnglish embeddings 
This 
means that we have bean more interested in the many forms of dependent 
14 
clau~e--~re~ositiondl phrase, relative clause, inf lnitive, participial 
phrase, relative cpnjuetive clause, etc,--than in the ffne detail on 
noun phrase, noun-noun combinations; and the fine grain of verb sttings 
We have also for the moment ignored ordinary conjunctions in view of the 
clear treatment offered by Woods, Wipograd and Grishman; each of whom 
points out that an and or an "~r" triggers a special subgrammar that 
attempts to find a structural repetition of a constituent that was just 
cbmpleted. Bqcause of our interest in embeddings we have chosen to consider 
relative clauses at the toplevel of the grammar where possible. 
The following constituent description defines a very fluid subset 
of English with great potential for embeddings. 
CLAUSE -, (NP) + (VP) 
-+ (DCLAUSE) + CLAUSE 
NP 4 ('APT) +. (ADJ*) + N + (DCLAUSE) 
-+ PRON + (DCMU$E) 
VP + VG a+ (NE') 
I? G - (AUX*) + (ADY) + v + (ADV) 
DCLAUSE -t PPI RELCON'JI~RELCLAUSE\~ VMOD 
PP + PREP* + NP 
BEUDNJ -, RCONJ + CLAUSE 
RELCMUSE -+ (RELPRON) + PRONCLAUSE 
PRONCUUSE -t VO( WP + ,VG .+ (DCLAUSE) 
vMoD +~PA~T/~P(~PR~~PART/vPIvINF/~ 
VPAST '+ SUPPORTED SAILED, . . 
'VINF -+ TO SUPPORT, . , . 
VPRESPART +.SUPPORTING, SAILING,.. 
RELPRON -+ WHO, WHICH,, WHAT THAT 
RGONJ -, BEFORE, AFTER WHILE 
AUX -+ IS WAS; HAS, HAVE, Hw 
ADV + HDRIZONTALLY. VERTICALLY 
V ' SUPPORT $ALAPJCE, SAIL 
PRON + F@, SHk, ET , THEY ., 
ART + A, AH, THE ... 
AD5 -+ LARGE, SMALL, TINT' 
1N + 'CLOWN. 'PEDESTAL, BOAT, DOCK,'FEET, TOP, SIDE .., 
In the a'bove: + means "followed by7, (x) means optional x. x* pedns 1 
\I 
or more 
means "or", . . . means etc. ,. md x/y means x is the 
lnitia "element of p. The arrow.+ means "defined by:'. 
The form of notation above-is a concise recurgive description fcr the 
ordering of constituents. It shows nothing abqt the semantics that may 
be included in the sptem, and the flow of control far parsing is not at 
all wbvious. Augmented Transition Netwcwk.graghs foLlowing Woods show the 
conditions,on elements of the sentence and the flow of control in terms 
of directed arcs leavlng*nodes in a two-dimensisned diagram of the grammar 
Even more irnpartantly, an ATN proviaes f~r the display of semantic 
operations that are to be undertaken on each const2tuent. The convention 
for drawing an ATN is to write conditional statements above the arcs, and 
operations below. 
For example : 
S ;= NP + VS-4- NP 
P,USH NP. PUSH VS PUSH NP,' 
fl 
GTR SUB 
(PUT(LAST(GETR V3) (PUT (LAST(GETR V). ) 
"SUBJ(GETR SUBJ) ) "OBJ [GETR OBJ) ) 
In this net, if the sentence begins with an NP, the PUSH NP will return 
Ehe structurk of an NP iq the * registei. At that point the registex 
SUBJect is set to 'that value. When a VString is analyzed. by PUSH VS 'then 
V is set to the value'VS returned. At this point further structure is 
bvllr-by PUTting on the verb's property lisk the attribute SUBJ with the 
value contained $n the register SUBJ. Similarly, when an OBJect NP is 
parsed, it can be added to the structure of V and,the value of S can be 
POPped-4.e. returned--as the register V>which will allow access to the 
property list of rhe verb on which the values of subject and object can 
be found by consulting those properties as in (GET (LAST(GETR V)) "SUBJ) . 
The function LAST is used in thi's exhmple to obtain the last element of 
a list. 
Notice this e~ample illustrates.that our general approach to recording 
vsemantic information is one of putting detailed information such as the 
arguments or cases of a verb on the property list of that object. Thus 
the result of parsing "clowns hold, poles" with the above net is: 
(HOLD SUBJ CLOWNS, OBJ POJ+ES) 
In,fact, it is necessary to create new names for each word used in a 
sentence--to avoid clobbering dictionary information--so the result from 
actual nets would be: 
(C1 TOK HOLD, SUBJ* C2, OBJ C3) 
(C2 TOK CLOWNS, NBR PL, DET INDEFj 
(C3 TDK POLES NBR PL, DET INDEF) 
Tke:- relation TOK shows that C1 is, an instantiation of'-the lexiix.11 item 
HOLD. In this convention for stating property list values, the first 
element is the ATOM and each pair separated by comas IS an ATTRIBUTE 
and i-ts VALUE. 
The Wdods system also btores it$ pa-st states and provides for backup 
in the event that no conditional arc succeeds and yet there is still 
sentence to be scanned., In this event the system recursively consults 
the state leading to the current node'to see-if there were arcs that wer.e 
untried that lead to a successful parsing for the sentence string. 
T-he 
* register has special sigdficance in that ordinarily it contains the 
sentence element under the scanner, except wheo a subnet such as NP 
returns a value, in vhidh case the POP arc sets the value in the * 
register. The overall flow of control through an ATN is that * is set 
to the first element of the sentence, then the topmost net, CLAUSE ot S, 
applies the grammar in topdown fashion. Each the a constituent --a 
word, a'phrase, a clause--is recognized and control is passed to another 
notie, the scanner' is advanced and parsing proceeds from the new node. 
For programing simple grammarg without much embedding and without 
backup capabilities 
a~1 ATW may be used a8 a flow chart to design the 
program. 
If more complex grammars are requhed, Woods has provided a 
complete set of language conventions -and gn Interpreter with the capability 
18 
of storfqg past states and backup. 
hxicdn t English wpsds; their lord claerres ad f eaturea and other 
information such ae program definitioos etc. are recorded on a property 
list structure for eady access by fun&tions used in the ATN. The follow 
ing examplee illustrate this structure: 
(cW (N T) (NBR SING) (EXPR (LAMBRA(). . . .)). . .(FEET XY) (ANIM T)) 
(BALANCE (u T) (%ENS@ PRES) (EXPR(LAMBDA(ST). . . . )) ) 
(ON (PREP T) (EWR(LAMBDA(N1 N2). . .) ) 
(WRO~(P~N T) (NBR (SING PL)) (PERSON T) (RELPRON T) ) 
The fuaction (PUT X Y 2)--e.g. (PUT "CLOWN "NBR ''SING)-YI~~ add the pair 
(Y 2) to the atom X or replace the value of X's attribute Y with the new 
value Z. The function (GET X P) will then return the value Z. Such ATN 
functione as CAT ahd GETP simply call GET tdth the first argument set 
to the value of the word under the sentence scanner. 
The l$XPR yalues associ'ated with an Engaish word are aedtic 
functions that are explained later. by modificaticms to this simpjrr? 
spheme can be aaded to provide for morphorogical variants referring-to 
root f om instead> of rhquiring a. definition oi-thef r own, and an 
attribute; POUOWEDBY, can be used to cbllact multiple ward terms. The 
basic property list representation of a dictionary can be expanded to 
include multiple word senees as well, but it always re,tains the character 
aE a basic LISP syetem for storage and retrieval of data aesaciated with 
an atom, 
/ / PUSH DCLAUSE SNT 
F 
PUSH VPfSENDR SUBJ POP (GETR HD) T 
HD + * 
(EVAL ('(GET * IITOK) *I 
TST CL1 (GETR £ID) POP (GETR m) (NULL Sl?TC). , 
. - 
Gltammar: This is thk toplkvel net for the grammar. I$ is named 
clause and transfers control to states C1 and C2 each of which can POP 
a value in the event that the sentence string has been completed or a 
clause successfully paased. The barred pointer,* , -indicates a HOP, 
operation which passes control without advancing the sentence scanher or 
changing the * register. 
This net accepts sentences beginning with an NP, a VP or a dependent 
clause. [.ID is the name of a register that generally contains the last 
constituent found. The UNHOLD arc emanating from C1 causes a list, HOLD, 
to be processed. HOLD contains Dependent Clauses that are missing soare 
element that delays their semantic processing. 
Phx example, "on his nose" 
in "on hie nose a clown balances" caanot be eema@,~fcally procesped until 
ll~lovnl' ehovs up as a following NP. 
The net ie satisfied by a sentence 
11 
or-by a single noun phrase euch ae a clown in n boat" dr by an imperative, 
"balance a pedestal". It ddee not ac'cept queetion forms; that would require 
an additional arc from CLAUSE labelled, PUSH QFOBN SNTC. 
The ordinary form of 
an arp is an arc-label such as CATegory, PUSH,, POP, TST followed by its arguL 
ment, followed by any-condition statement. 
SNTC is simply the variable that 
20 
contains any remaining sentence string, so the condition SNTC is true 
except when thelstring has been..exhausted. If SNTC is nil, there is no 
point in further processing. 
The arc PUSH W (SENDR sUBJ)~will send the value of the register 
SUBJ to the qubnet W.* If VP is SUCC~SS~~~, the operation under the 
qrc (EVAL-((GET * TOK) * )) will caJ1 for a function associated with the 
verb to translate the subject, ~bject and complements of the sentence 
into the particular semantics of pictorial relations. The verbs SUPPORT, 
SAIL, and MOVE are defined as semantic functions in seetion V. PreA 
positions are also defined as semantic functions in that section. 
When HD is popped from C1 or C2 it contains the name of an object 
ondthe property list as described earlier. The resblt of a parse is an 
atom name whose property list contains labelled references to its 
arguments which are either symbolic or numeric values, or references to 
other atoms which have property lists. ~h"is of course is a property 
list representation of a semantic network. 
CAT PRON 
I' 
C 
h~ f- (ANTEC * GLST) 
1 DET + DEF I 
(NP~) POP HD (PUT HOI .I 
TCT CIY T 
CAT NI 
DET 4 INDEF 
HD (MAKETOK*) 
PUT HD "DET DET 
"MOD MOD 
* SENDR is usually signified in the nets by ). Thus + SUBJ meansq 
(SENDR SUBJ (GETR SUBJ)) X + Y meana (SETR X (APPEND Y (LIST (CETR X)) ) ) 
This NP net is operated dh the tall, PUSR NP T. 
It allows for a 
prono--or a sequence of (arr)(adj*) N. Its operation intludes some 
basic semantic transformations on the head nbun. If the sentence begins 
with an-ARTtcle, the determination is set to DEPINITE or INDEFINITE 
depending on what feature GETF finds associated wdth it. A pronoun 
implies definite determination, and a noun phrase withouban article 
implies indefinite exceptdn the case of proper nouns not considered in 
this net, Adje'ctives are appended to a list named MOD. 
When the noun head is encountered, MAKETOK creates an atomic name ~i 
using the LISP function (GENSYM C) and puts on its property list, the 
pair, TOK WORD. The remaining operations under the CAT N arc add property 
value pairs to this TOKen of the noun. From NP2 the atc, POP HD 
(PUTMODS HD), is encountered. PUTMODS is a semantic function that works 
with adjectives and adverbs iri the following fashioe: 
An adjective, e.g. big, has the following lexical structure: 
(BIG ADJ T, POS T, TYPE SIZE, VALUE 7) 
PUTMODS will for each adjective obtain the TYPE and VALUE and put them on 
the noun's property list. Thus, "a big red clown" results in: 
(C1 TOK CLOWN, DET INDEF, NBR SING, SIZE 7, COLOR 1) 
where COLOR 1 assumes that some mechanism for assigning colors likes numbers 
as inputs, even as the drawing programs require numerical values for 3IZE. 
The result of parsing a noun phrase with thismetwork is to return the 
semantic structure of an object as a set of property-value pairs associated 
with the name Ci which is a tbken of the word used. The net is not sophis- 
ticated as NP definitions go, much more complete grammars of the NP are 
offered by Winograd and Woods. 
The lack of a continuation into a modifying 
22 
clause such as a PP ordrelaYive clause is deliberate in that we prefer 
to rbturn control to the structure calling the NP so that its syntactic- 
semantlr position in the higher sequence can be used by the Dependent 
Clause net. 
'PUSH NP SNTC 
r PUSH PP "BY PASV 
OBJ + 3 
PUT Vt'SUBJ SUBJ 
SUBJ +- * 
PUT v !ISUB-J SUBJ 
'if "OBJ'OBJ 
PUT Y "OBJ OBJ 
TST AUX=BE V=ED POP v (PUTI v I'SUBJ SUBJ) 
PA$V + T 
OBJ+ SUBJ 
SUBJ t NIL 
t 
This VP net first pushes a VG, verb group. VG is not shown in this 
discussion, but it scans the sentence string for an acceptable sequence 
of auxilaries, and adverbs domindt'ed by a verb. It makes a token of the 
verb and puts its tense and auxidiaries on that token as property value 
pairs. It returns the token name. In exiting node VP1 we seek an NP as 
a syntactic OBJect and finding one, add the subjgct and object as properties 
of theverb. If no NP folloys the verb, the next arc tests to debermine 
whether the verb is a passive form and if so sets the flag PASV, sets 
object to subject, and subject to nil. If a "by" prepositional phrase 
follows, it becomes the subject. Additional modifying phrases are picked 
up by the DCUUSE loop. No actlons are associated with PUSH DCLAUSE arcs 
23 
because each DCLAUSE calls semanti,~ routines that bind the modifier to 
the noun or verb it modifies--frequently not the one it ilnmediately 
follows * 
The YP net accepts a verb, a verb group, or a verb group followed by 
an NP and a string of PPs or other modifying clauses. 
It lacks the case of 
two NPs to account for direct and indirect objects. 
PUSH PI' (GAT PREP) 
h rl 
HD + * 
A 
POP HD T 4 
PUSH RELCONJ (CAT RCONJ) 
TST GETR SUBJ 1 
CAT RPRON @ PUSH, PRQNCLAUSJ) 
(.EVAL ((GET * T~K) * ) TST 'I 
I 
SUBJ + (ANTEC * GLS? 
((HOLD *) 
J. SUBJ 
CAT v "ED "TNG 
P 
11 TST T 
SUBJ + (VBMATCH * GLST) 
p, 
HD +- 
-I. SUBJ 
* = "TO. NEXT = V 
The DCLAuSE.net is fairly intricate in that it accounts for PPs, 
relative pronouh clauses, infinitive modifiers, participial c Wses and 
clauses introduced by relative conjuncti~ns. A PP is we or more prepositiotrs 
followed by an NP. A RELCONJ starts with an RCONJ such as "while", "after1' 
etc. and may be followed by a DCLAUSE or a CLAUSE. 
A relative, pronoun 
clause begins with an optional relative pronoun and is followed by a pronoun 
clause which is either a VP or an NP fohlowed by a VG an6 optional DCLAUSES. 
For the moment we insist for computational economy that a relative clause 
he introduced by a relative pronoun; actually the fm of a pronominal 
clause is sufficiently rwell defined that PUSH PRONCLAUSE can identify it 
without a relative pronoun -in most cases. 
When a pronoun is found, here or in an NP, the- function ANTECedent 
is called to scan the list of prece'ding nouns, to find the best agreement-in 
person, number, and gender. The function VBMATCA on the exit from node D2 
is a function that seeks to find the head that the participial or infinitive 
phrase is modifying. 
As in PREPMATCH, the head noun is frequently not the 
one just preceding the modifying phrase and-the particular verb and its 
ending are usedl ip choosing its head nauh or verb. GLST is the name of a 
list of candidates. 
In 'the event that the DCLAUSE is a relative pronoun or a participial or 
infinitive construction, the final step is to call the semntic function 
associated with the verb and evaluate it for the subject, object and 
complemeht arguments. DCLAUSE is undefined for adjectival and adverbial 
clauses that can be used~as modifiers. When defined they can be added as 
additional arcs. 
CAT PREP "NEXT U 
PUSH NP (GETR PREP*) POP Hb (PREP 4- NIL) 
PUT "PREP PREP 
HEAD + * 
(PREPMATCH * GLST) 
This abbreviated PP net is presented to call attenti~n to its method for 
accepting a ~tring of prepositions and for accomplishing the semantics by 
calling PREPMATCH. Although Section IV concerns semantics, it is warth 
noting that the eftect of PREPMATCH is to add information to the semantic 
structure reptesenting a noun or a verb. For exa~ples: 
"a clown on a pedestal on his nose 
I @ 
(C1 TOK CLOWN*, SUPPORTBY #PEDESTAL, BALPT #NOSE) 
I1 
... balances on a pedestal on his nose" 
(C2 TOK BALANCE, TENSE PRESENT, COMPS (IPE~ESTAL #NOSE) ) 
Thus if a verb intervenes beween a voun and prepositional phrases that 
might ntodify it, the PPs become COMPlements to the verb under the attribute 
COMPS, and the verb's semantic function has 
the task of relating it to 
other elernems of the sehtence. 
V Semantics of the Subset 
Parsing a sentence with the ATN grgmmar just described results in a 
get of symbols edch of which is further characterized by attribute* and 
values on a property list. If no semantfc functions were applied--such 
as those associated with prepositiwns, modifiers and verbs--the result 
would be a tree such as the following: 
(C1 TOK BALANCE, SUBJ (C2 TOK CLOWN, DET DEF), 
OBJ (C3 TOK POLE, DET INDEF) , 
COWS (C4 TOK HANDS, POSSBY C2, PREP (XI)) 
ie effect a£ the sernaotic functions for this sentence is to produce the 
f t llowing : 
(C2 TOK CLOWN, SUPPORT C3, SIZE 3, ATTACH ((22 C3 )) 
XY XY 
(C3 'OK POLE, SUPPORTBY.C2, SIZE 3, ATTACH (C-3 C2 )) 
xy SY 
which is minimally sufficient information for the graphics to produce a 
single icture to represent the state of affairs the .sentence described. 
It 
s perfectly feaslble to compute the syntactic form first and then 
apply the semantics, but as Winograd, Riesbgck andmothers have found, the 
early application of semantics can be used to minimize the ambiguities of 
the syntax. For this reason, as each prepositidnal phrase is parsed a 
semantic function is called to determine which noun or verb might be its 
governor or head. Each time a Verb Phrase is completed, a sema~tic 
function is called to translate its syrltactic arguments, i.e. SUBJ, OBJ, 
COWS, into pictorial relations such as SUPPORT, ATTACH points, etc. 
Semantics of Prepositiops: After a PP constituent has been identified, 
a function PREPMATCH is called with a list of the nouns and verbs so far 
encountered, GLST.! Each preposlition is associated with a function that 
examines a candidate head from GLST-hnd the naun object to decerpine if the 
candidate can dominate the PF in question. For example "ON1' is defihed as 
a LISP function with two arguments. When called with '"clown'! and "nose", 
ON returns a structure in which the ATTACH poifit of the clom i's the XY 
coordinates of his nose. When called with "clown" and1"pedestal" it 
returns a structure in which the pedestal SUPPORTS the clown. If dallad 
with "nose" and "pedestal" it returns NIL sihce nose is neither- ah inde- 
pendent picturable object nor a paft of the pedestal. 
PREPMATCH does the book-keeping by calling the preposition function 
with each candidate from the GLST, If*he candidate is a verb tlha~ 
can be modified by that preposition, PREPMATCH adds the PP to the verb's 
list: of'C0@S, and the verb gemantic function will interpret it. Tlie 
function BESIDE offers a simple example definition that shows how dne 
prepositien can imply another. 
(BESIDE(LAMBDA(N$ N2) (RIGHTOF N1 N2) )) 
(RIGHTOF (LAMBDA (N1 N2) 
(CO'ND ( (AND (GET N1 "PICT) (GET N2 "PIcT) ) 
(PUT N2 "RIGHTOF Nl) (PUT N1 "LEFTOF N2) ) 
(T NIL) 1 1) 
Thue. "a beside b" is quite arbitrarily interpret& to mean "b ie ~o the 
right of a". 
RIGHTOF requires that its two arguments be picturable 
objects. 
"A clown on his nose beside a pedestalr' causes PREPMATCH 
((NOSE, CLOWN) PEDESTAL). PREPMATCH first-calls (BESIDE NOSE PEDESTAL) 
I I 
BESIDE calls RIGHTOF which returns pecause nose" is not an independent 
PICTure. Then PREPMATCH calls (BESIDE CLOWN -PEDESTAL) and the return is 
(essentially*) PEDESTAL RIGHTOF CL,OWN. 
Somewhere else in the forest, the relation RIGHTOF will be interpreted 
to mean contact between leftsi.de and'rightside of two objects. So we.quite 
arbitrarily' force a.p.lte~ise meaning--so far suff&ci~rit for our purpose-- 
om the geometrically vague term, "beside". In general the prepositional 
semantics for a micrpworld model are definable where the number of possible 
meanings for each preposition are limited by the situation. ~h the CLOWNS 
wor Id, "with" "on" and "by" have multiple meanings that are selected .in 
accordance with the conditions described by their semantic functions. 
In .contrast, "from" so far has a single meaning. 
Verb Semantics: The English verb is a remarkably complex conceptual 
object. It may carry several aeanings dependent on its arguments and on 
its larger con-text. It communicates information about temporal ordering 
of its process by auxiliaries and its suffix. It implies one or a sequefitial 
seriee of events. Its syntactic positi6n and ending can be used to signal 
that it is a pre-modifier or a post-modifier for another verb or a noun. 
It is part of a clasqification structure and may imply special argument 
valuee to some more general verb higher in the classification. Far example. 
* Where these examples use words trhe functions are using Ci tokens or 
words appropriate. 
2 8 
"retort" means '.'answer sharply" ~hich means "comutlicate sharply ii~ response 
to a communi.cationt'. The verB mdy imply Gpecial arguments in another way; 
the verb, "sail", implies that "someone caused a vehicle ta move through a 
fluid by a means involving aerodynamics from 'one place to another" If 
the sentence omits some of these arguments, the verb semantics implies 
them. Thus we can sail a boat, a kite, an airplane, a saucer, but hardly 
a locomotive or a desk. If the arguments are idappropriate we can ascend 
the classification tree and call the statement a metaphor. In addition, 
the verb allows its arguments to occupy practically any syntactic position 
in the clause or sentence and must sort theui out oa the basis of semantic 
informat ion. 
By analogy, a verb is a dramatic skit with a variable set of characters 
that successively relates the character roles to one another over a period 
of tjme. A verb has a set of a'rguments, case roles filled by semantic 
objects; it has an initial state, a set of relations among its characters; 
a set of intermediate states, one or more sets of relations among its char- 
acters; and a final or resulting,,state similarly charafterized. In addition, 
receht work particularly by Abelson and Schank suggest that in a given 
culture a verb models a situation that is predictably preceded and followed 
by rntrre or less typical situations. If a person strikes another person,,the 
first one was probably angered by the second, dominates the second, etc. 
while the second, feels pain, map react in anger, etc. So it is reasonable 
to scppose that our experience is organized in scripts, frames, scenes, 
dremes, etc. whose component elements include the dynamic skits that verbs 
signify 
In the CLOWNS world a verb selects an associated semantic function to 
sort its arguments into typical roles in its picturable dr.amatXc skit and 
relates them in typical ways for display as initial, intermediate and Final 
conditions. 
In this rashion, the verb "sail" relqtes an Agent, a Vehicle. 
a Medium, a Start point., Inmrmediate points, a Goal point and possibly a 
Means of movemant. #The semantic routine must translate syntactic entities 
such as Subjec~ Objecr ana Complements into these roies, i.e. bind the 
variables. It must then relate them'iii iippropriate ways-- AGENT'IN VEHICLE. 
VEHICLE AT STARTPOINT, VEHICLE'ON MEDIUM, etc.--for each of its temporal 
states and call the graphics sysrern to display them. 
Support is a. verb that describes a static Single state of af€a.irs in 
"The world is supported on a turtle's back". The verbs "balance", '.'support", 
''stand'' "huld". are each as-sociated with the semantlc tunct ion SUYPUK'I'~. 
When a VP constituent using one oi these verbs is completed, SUPPORT1 is 
called to compute a m,odel of the situation described. 
SUPPORT1 binds the cases TH1, TH2, SUPPORTPTl, BALPT2. TH stands for 
THEME and the other two cases ror Support Point, and Balance Point. The 
following diagram shows the spatial rela tion:. signifie.d by these cases: 
Thl supports TM2 on its BALPT2 on/with/in his SUPPORTPTl. If these four 
arguments are bound. the suqport relation is completely defined. 
If not. 
means are taken to till in. the missing arguments by 2 default logic. 
SUPPORT1 takes as arguments, SL5J, OBJ, and CflMPS where COWS is a list of 
complements. SUBJ and OBJ were computed by the VP parser as the subiect and 
object of the ACTIVE £om of the clause. 
The conditions or rules For transforming these syntactic arguments 
* 
into semantic roles are as Collows: 
SBBJ A OBJ + mi +- SUBJ. TH~ 4 OBJ 
SUBJ -+ TH2 + SUBJ 
OBJ + TH2 + OBJ 
For each ZOMP, 
NTH~ A ON A PSCT(C0MF) -+ TH1 +- COMP 
nlSUPPORTPT1 A IN v ONVWITH A PART(C0MP THL) + SUPPORTPTl + COMP 
dBALPT2 A ON A PART(C0MP TH2), -+ BALPT2 + COMP 
T + PRINT (LIST "UNDEFINED COLON COW) 
For the following two example sentences, the above rqles result iri 
the bindings (shown: 
Ex 1 A c,lown balances a pedesta1.0~ his head on its side 
I I I I 
TI11 TH2 SUPPORTPTl BALPT2 
Ex 2 A clown balances on z! pedestal on-its side on his hegd 
I I I i 
Additional modifiers may have been present as-in the example sentences: 
A clown, on his hands balances a pedestal on his head, on its side 
beside a pole. 
A ololn with a pole in his hands b'alances on a pedestal.. . 
The earlier action sf rhe preposition semantic functions will have reduced 
these additional complements to no more than those ~bown in Edamples 1 and 2. 
'Notes:' x-+~y X imp'lies Y 
x 4 y SET x to y 
Ax Not X 
F'(x) Evaluate function F of X 
n( and 
V or 
" Quote 
Brief forns such as "A clown balances on hi$ hands" or "A clown 
holds a pole" 'result in inbomplete bindings from the rules of SUPPORTL. 
The legitimacy of suqh brief forms iequires a default logic that in the 
first case assumes that the Ground supports the clown-at a point called 
TOP of the ground. In the second case, the clovn.'s SWP~RTPT~ for 
the pole 
Is bound to hrs hands and the BALPTZ--for the pole-- is bound to the 
BOTTOH of the pole The verb "hold" puts a default value of "hands" on 
the structure it passes to  SUPPORT^ according to the following definition: 
(HOLD (LAMBDA (ST) 
(PROG ( ) 
(PUT ST ''SUPPORTPT~ ''HANDS) 
wm~~ (SUPPOBT~ ST)) >I) 
The default logic af the verb seeks these values to bind them appropri- 
ately to any-empty case arguments. The more general default values of 
TOP as a missing SUPPQRTPTl and BOTTOM as a missing BALPT2 and the fact 
that the object on the bottom of the heap must be supported. by the GROUND 
are all supplied just prior to constructing a picture frame. 
The result of SUPPORT1 is to create a process model of the fbllowing 
form: 
(Ci TOK balance, GLOBAL (...)-,INIT(...),INTER(...) 
RESULT ( . . . ) ) 
The value of the attribute, GLOBAL is a quoted set of (PUT X Y Z) whrch Me 
true at all times in the model. INIT !is the set of relations true Bt the 
initial state of time 5n the model, INTER Ts those for the intermadiage 
states, and R~SULT is the set fbr the final state. When a functionWG 
for Pragmatics evaluates one of these attributes, the result is to evaluate 
these PUT functions to produce a semantic network representing the state of 
32 
affairs at a given instant of time. The semantic relations are translated 
to ATTACH 4-tuples which then generate a picture of the state. Successive 
pictures are obtained by calling PRAG repeatedly for INITial, INTERhediate, 
and RESULT states. 
For the examples of the SUPPORT1 verb, only the GLOBAL attribute is 
given values as f~llows: 
ci . . . ,"GLOBAL (LIST ("PUT mi "SUPPORT m2) 
("PUT TH2 "SUPPORTBY TH1) 
("PUT TH1 "sUPPO~PT SUPPORTPTl) 
("PUT TH2 "BALPT BALPT2) ) 
Initial, Intermediate and Result states are null since the verb simply 
describes a static state. 
The verb MOVE* is more,complex and more interesting. Let us assume 
I I 
as input the-sentence, A clown on his head ;ails frorn'Corno to Menaggio" 
Wheh the parser has completed its VP the semantic structure.1~ as follows: 
(abbteviated to the portion relevant to-this discussion.) 
(Cl TOK CLOWN, BALPT HEADXY, SIZE 3) 
(C3 TOK SAIL, SUBJ C1, COME'S (C4 C5), TENSE PAST) 
(C4 TOK COMO, . . . ,PREP FROM) 
(C5 TOK KENAGGIQ, ..., PREP TO) 
At this point VP calls'(Sh1L C3). SAIL is defined as f~llows: 
(SAIL (LAMBDA (ST) (PROG ( ) 
(PUT ST lq~~~~~ "WATER) 
(PUT ST "VEHICLE "BOAT) 
(RETURN (MOVE* s T) ) 1)) 
That is, SAIL implies a movement of*a boat on water and so passes thiS 
Lnformetion to mVE* which may have to use it to bind lte case roles of 
5EDIUM and VEHICLE which in fact at? not mentioned explicitly in the 
example sentence. 
MOVE* binds the arguments Agent, THeme, VEHICLe, - Source, Goal, and 
MEDiuhby sorting out the information contained in SUBJ, 0EU and COMPS 
by the following rules: 
ANIM (StJBJ) + A + SUBJ 
FORCE (SU3J) + 1 + SUBJ 
VEHIC (SUBJ) -, VEHIC + StJBJ 
S VMIC /\ mIC (ow) + WBIC + OBJ 
MEDIUM (OBJ) + MED + OBJ 
OBJ -+ TH + OBJ 
FOR EACH COhF 
a MED A IN V ONLV THROUGH A MEDIM(CObfP) + MED + COMP 
% VEHIC A IN V ON V WITH A VEHIC(COMP) + VEBIC + COMP 
% S A FROM fi PUCE(C0MP) -+ S 4 COMP 
G A TO 4 PLACE(COM2) -t G c COMP 
T + PP,~ (LIST lr~~~~~~~ : COMP) 
DEFAULT : 
s VEHIC + VEHIC + (GET ST WHIG); % 6 -+ S + (MAKETOK "POINT) 
.\I MED + MED + (GET ST MEDIUM) ; w G -+ G c (MAKETOK "POINT) 
Thie definition of the canditione for MOVE* is atill incomplete 
except fox the verb "sail" and will be modified with further experience. 
Haying bound the role varieblee, MOVE* creates a procese model by 
assigning to ST, gets of value8 for the attributee GLQBAL, INITlal, 
INTEBmediate, and,RBULT. 
34 
For GLOBAL conditions, 
(AND I (PUT S "SWPORT 1) (PUT MED "SUPPORT VEBIC) 
(PUT' I "SUPPORTBY S) ) (PUT VEELC "SUPPORTBY MED) 
(AND A TH (PUT A "LEFTOF TH) (PUT VEBIC "SUPPORT A) 
(PUT TH "RIGHT OF A)) (PUT A "SUPPORTBY VEZIIC) 
(AND A (PUT mnIc 11~~~~~~~ A) ) (PUT MED "LEFTOF G) 
(m TH (NULL A) (PUT VEHI C "SUPPORT TH) ) -(PUT MED "RIGBTOF S) 
For INITIAL, 
(PUT VEHIC '1~~~~~~~ S) 
For INT~~mediate? 
(REMPROP TJMIC '"RIGHTOF) 
(REMPRO? s "LEFTOF) 
(PUT VEHIC "BETWEEN (S G) ) 
Por RkSULT, 
(REMPROP VEHIC "BETWEEN) 
(PUT VEHIC "LEFTOF G) 
(PUT G "~IGHTOF VEHIC) 
Ffg. 3 shows theee states in the form of a process model, 
When this process model, C3, is evaluated, the function PRAG is called with 
the arguments C3 and either INIT, INTER, or RESULT. PRAG will first inter- 
pret the GLOBAL attribute causing the state represented on the property 
liste Tor Tokens of clown, boat, etc. to be changed. It will then make 
the changes indicated by the PUT8 which are additions, and the REMPROPs 
which are deletions. If PUG ie called three times in succession for INIT, 
INTER, and RESULT, three euccessive sfatee are created to shaw the pro- 
greeeion of the process from etart to finieh. After PRAG has been called 
the support points and balance pointa are all defaulted as neceeaary to TOPS 
and BOTTOMs by the function that calls the GRAPHIOS system, This function 
I SWPORT 
INIT: 
PUT VEHIC "RIGHTOF S 
INTER: 
~ROP VZHIC  RIGHTOF OF 
RESULT : 
PUT VEHIC   LEFTO OF G 
Figure 3. PRocess Model For,MOVE* 
also establishes horizontal contact points for BETWEEN, RIGHTOF and 
LEFTOF. 
VI Semantics of Scenes 
A scene is composed of a set of Pictures related to each other by 
adjacency and Support relacions including their poiats of contact. A 
picture is a LOGO display program that when called with a given start 
point and heading of the display turtle or cursor will construct a two 
dimensional line drawing. A square can be drawn by the following sequence 
of operations. (See Papert 1972.) 
PENDOWN, FORWARD 20, RIGHT 90, FORWARD 20, RIGHT 90, 
FORWARD 20, RIGHT 90, FORWARD 20, RIGHT 90, PENUI?. 
The last ''FIGHT 90" restores the cursor to its original heading. FORWAEU 
and BACK axe vector making functions that draw a vect'or from the current 
xy point of the curser a given number of +its in the direction the cursor 
is aimed' The language uses functions with arguments and may create and 
call subroutines. Square may be defined as 
SQUARE;SIZE: FORWARD :SIZE,. . . -.ETC. 
If a triangle has als~ been defined, we can then define: 
H0USE:SIZE; SQUARE:SIZE; F0RWW:SIZE; TR1ANGLE:SIZE; 
It is the convenience and simpl~cityeof these LOGO conventi'ons that 
convinced me that drau3,ng pictures from sentences would not add any gteat 
complexity to a basic language analysis system.. LOGO offers many additional 
features as a language for teaching programming skills to non-mathematically 
oriented users and one of the most important of these may be as a parenthesis- 
free form of LISP. 
In our use of LOGO graphics, we-consider that a picture has a name, 
a program to praw it, a cursot startpoint~value, a head'ing, a size, a 
frame of minimum arid maximum X and Y coordinates, a center of gravity and 
coordinates associated with any points on it that we need to refer to, such 
as feet, hands, head, top, bottom, etc. 
CLOWN EQR (LAMBDA.() ...I 
SIZE 1 
STARTPT (XY) 
HEADING NBR 
PFRAME (MIN X, MAX X, MIN Y, MAX Y) 
FEET (Xi) 
a 
a 
BOTTOM (=> 
All of the XY coordinates designated in a picture structure are relative 
to the startpoint, heading and size. If we set the startpoint to a given 
value, say 500; 0; the clown will be drawn from the bottom center of the 
screen. If we set HEADING to 90, it will be drawn on its side. 1f we 
change size to 2 each vector composing the picture will be twice as long. 
If we whsh to translate the clown to the right 50 units, 50 is added 
to the X coordinate of the startpoint. IF we wish to m6ve it up, a 
number is added to the Y coordinate of the startpoint. If we Wish to 
rota-te it onto its head with its head at 500,100 life is more difficult. 
We must use trigonometrikc functions to compute a heaaing value and a 
location of the startpoint that will achieve this result. A functipn 
called ORIENT* takes as arguments an object, i-ts balance point, and a 
reference point. 
(ORIENT* CLOWN, HEADXY , (500,100)) 
This function adjusts the startpoint and heading so that the head of the 
clown will be at (500,100) with the center-of gravity above the point. 
Siinilar- adjustments are made to the PFRAME values to translate and rotate 
the imaginary picture frame defined by the XY extremals. 
To assemble a set of pictpres into la scene, the bottom pieture is 
assigned an XY s~tartpoint and heading. Each picture it supports is 
translated and rotated to result in adjustments to startpoint, heading and 
pframe values. Each picture beside it is s~ilarly adjusked until a 
scene is cgmpleted by accounting for all its pictures'. At tQis point, the 
scene is scaled to the size of the display screen, and the picture drawing 
programs are,executed. 
The PFRAME concept developed by Gordon Novak and Mike Smith is very 
helpful as a computafional abbreviation Eor.fhe program that draws the 
picture. The PFRAME attribute has a minimum x, maximum x, minimum y, 
maximum y as four points that define a rectangle that surrounds the extreme 
points of the pictime, When the picture is programmed these are assigned 
by hand with reference to whatever startpoflt and heading were used. The 
picture as defined is taken as size 1. Whenever the picture is translated 
or rotated the values of PFEUME, STARTPT apd HEA~ING are adjusted accordingly. 
As each pair of piccures are combined into a scene, a FFW is computed 
for the scene. The final PFRAME for the entire scene is adjdsted to the 
size of the screen with appropriate scalidg of the size values of its 
component pic turas. 
39 
A f'requent use of PFRAMES is to find default values for TOP, BOTTOM, 
LEFTS~DE and RIGHTSIDE as contact points between pairs of pictures . 
Dep'ailed descriptions of these processes are-not particularly relevant to 
this paper's goal of presentipg an easily computable syntactic-semantic 
scheme for subsets of English but will be presented in forthcoming papers 
by Bennett-Novak and by Michael Smith. 
VII CONCLUDING DISCUSSTON 
In previous sections the terms "proceas madel", "skit", "~cene'~ and 
"pf rame1' have been wed to describe very llmited' structures of verb and 
noun semantics. This usage is in contrast to the much broader ideas 
associated with "scripts1', "f tames" etc. which ate typically used to 
describe worlda of vision ma belief system&. Example process models 
for "support" and~'@mve1' have been described and applied to the task of 
organizing images- into scenes. Nouns such as "clown" , "dock", "pedes tall', 
etc. have been represented as programe that construct line drawings. 
Adl\ectives have been used to communicate variations in eize, and adverbs 
to\ indicate angles. Other nouns, such as "top", "bottom", "edge" etc. 
are defined as functions that reference p/r ticular x-y coordinates of s 
picture. 
I1 
Npuns such as circu~", "party", "ballgame" etc. have not yet been 
attempted. 
They imply partially ordered sete of proceas models and are the 
most exciting next step in this research. 
More complex verbs like "return" 
or "make a roundtrip" imply a sequence of interacting proceas modele. 
Thus. 
"a clown sailed from the lighthouse to the dock and rethrned by bus" offers 
40 
interesting problems in discovering the arguments for MOVE*-return as well as 
in the design of a higher level process model whose intermediate conditions 
include the models of MOVE*-sail and MOVE*-return. 
We have also noticed that the semantic network that is produced as a 
result of semantic analysis can be seen as a problem graph by the functions 
that organize images and it is apparent that as these graphs come to contain 
larger numbers of images, it will be necessa'ry to d'etrelop graph searching 
etrategies along the lines of ordinav problem solvers. Our first experiment 
in this lihe will be to semanti~allp analyzeo the miseionariee and cannibals 
problem and illustrate the solution. 
As it stands, the CLOWNS system has served as a vehicle for developing 
and expressing our ideas of how to construct a tightly tntegrated language 
processing system that provides a clearcut syntactic stage with coordinate 
semantic processing introduced to reduce ambiguity. Twd stages of semantic 
processing are apparent; the first is the use of prepositions and verbs to 
make explicit* the geometric relations of "support", "lef to£" etc . among the 
objects symbolized by the nouns ; the second is the transformation of these 
geometric relations into connected sets .of x-y coordinatee that can be dis- 
played as a scene. Schank'~ notion of primitive actions is refle,cted in our 
approach to programming high level Verbs such a8 MOVE* to encompass the idea 
of mation carried in verbs such as "sail", "ride", etc. Yoadd ATN approach 
to syntactic analyais ie central to this system and in sharp contrast to the 
approach of Schank and Riesbkck who attempt to minimize formal syntactic 
processing. Our procees model reflects the ideas developed by Bendrix in 
his development of a logical structure 5sr English semantics. 
41 
The system is not li'mited to its prekent grammar nor to its preeent 
vocabulary of images. 
Picture programs to construct additional objects are 
easily cbnstructed and the semantic r~utihe6 for additional verbs and prepo- 
sitions can be defined fo the eystern wfth relative ease. 
We hope in the 
near future to illustrate the following sentence: 
"One of the ffrst plant8 to appear bn a newly formed volcanic PsJsnd is 
the stately and graceful cocbnut palm." . This will involve programming the 
11 
verbe, 
appearw', "fgtm", "grow", and programming pictured of plantg, coconut 
palm and islands. Very interesting problems are apparent in understanding 
and representing the ideas of "first" and "new" as well a~, in the relation 
between "plant en and "coconut palms". 
The system has been used successfully to communicate methods for natural 
language compu.ta#ion to graduate students and to undetgraduatey. 
It appears 
to have immediate possibilities for teaching the structure of English, for 
teaching precision of English expression, iind for teaching foreign languages 
through pictutes. Eventually 'it map be useful in codunction with very good 
graphic systems for generating animated illustration8 
for picturable- text. 
In my and CLOWNS sh~ws thk power and vaue of the mic,roworld approach 
to the study of Artificial Intelligence. 
By narrowing one's focus to a tiny 
world that can be completely described, onen can define a subset of ~n~lish 
in great depth. 
This is in corrtrast to the study of text where the situations 
described are so complex as to forbid exhaustive analysis. 
The translation 
into a visualized microworld provides an immediate display in a two-dimensional 
language of the interpretations dictated by the -syntactic and semantic systems 
and thus a scientific measuring instrument for the qccuracy of t&e interpre- 
tation. 


REFERENCES 
Abelson, Robert P., "~orlcepts for Representing Mundane Reality ic Plans. 
11 
In Representation and Understanding: Studies in Comitive Science, 
edited by D. Bobrow and A. Collins ~radedc Press. 
In Press. 
Badre, Negib A., "computer Learning from English Text." ~lectronics 
Research Laboratory, College of Engineering, University of 
California, Berkeley, 1972. 
Bobrow, D., and Collins, A,, Studies in Cognitive Scienck, Academic 
Press, New York 1975. In Press. 
Bobrow, D.G. and Norman, D.A., "Some principles of cognitive systerna, 
I I 
In Bobrow and Collins. 
Bruce, wrtram C., "A model for temporal references and its application 
in a question answering program." 
~rtificial Intelligence, 3 1972 
pp 1-25. 
Charniak, Eugene C., "Toward a Model of Children's Story Comprehension." 
A1 TR-266, MIT, Cambridge, Mass., 1972. 
Collins, q., and Wafnock, E., "Reasoning from Incomplete Knowledge. 
11 
In Bobrow and Collins. 
Fikes, R.E., & Nilsson, N. 3. "STRIPS: A new gpproach to the application 
of theorem proving to problem sglving, 
1 t 
Artificial 
Intelligence Vol. 2 pp 189-208. 
Gridhman, R., Sager, . Raze, C. and Bookchin, B., "The Linguistic 
String Parser." AFIPS, Conference Proceedings Vol. 42; MIPS Press, 
Muatvale, N.J. 1973, pp. 427-434. 
Harris, Larry R., "A Model for ~daptive Problem Solving Applied to 
Natural Language Acquisition." Thesis MSS, Department of Computer 
Science, dornell University Ithaca, N.Y. 1972. 
Heidorq George E. -"Natural Language Inputs to a Simulatfon Progradng 
System. " NPS-55W, ~Hyal Post Graduate Schoor, Monrerey, Calif. 
1972. 
Hendrix, G.,   rel lid nary Consfructs $or the Mathematical Modelling of 
English Meanings " 
Univers$ty of Texas, ~e~arthent of computer 
Sciences, Working Draft, April 1974. (not for distribution) 
I I 
Hendrix, C.G., Thompson, Craig and Slocum, Jonathan.- Language Processing 
It 
via Canonical Verbs and Semantic Models, Proc. 3rd Int. Jt, 
Conference on Artif ic.ial .Iatelligante, Stanford Research InsLitute, 
Mehlo park; Calif., 1973. 
Hobbs, Jerry R. "A Model for Natural Language ~emantics." Department 
of Computer Science, Yale University Research Report 836, 1974. 
Matuszek, D., and Slocum, J., "An Implementation of the Augmented 
Transition Network System of Woods." Department of Computer 
Sciences University of Texas, Austin NL-9 2972. 
Minslcy , Marvin, , "A Framework for Representing Knowled$e, " In The 
Psycholo~y of Computer Vis.ion, edited by P. Winston, McGraw Hill 
1975. 
ne 
winston, 
%pert, S., "~efich-lng Children to be Mathematicians vs . Teaching About 
Mathematics.'' Int. J. Math. Educb. in Science 6 Tech., New Y~rk: 
Wiley & Sons, lm; MIT, A. I. Memo # 249, July 1971. 
Rieebeck, C. K., "The Conceptual ailalyzer." In Schank, R. Conceptual 
Inf ormatYon Proceesing. 
Rumelhart, David E, , "Notes on a Schema for Stories. 
t I 
In Bobrow and Collins. 
Schank, R.. & Jibelson, R. "Scripts, Plans and Knowledge. " JGS, 
Yale U~iversity, 1975. 
Schank, Roger, Conceptual InIomtion Processinq, North-Holland 
PublishAng Company 19 75 (In Ptess) . 
Scragg , Gteg W. ,"LUIGI: An -EngTish Question Answering Program. I' 
Thesis MSS, AP & IS Dept., ~nidersit~ of California, San Diego 
1973. 
Sikl6ssy, L. & Dreussi, J., "An Efficient Robot Planner that Generates 
-. 
its om Pr~cedurei;.'~ Proceedings 3rd. Iat. - 
Jt. Conf . on A. I., 
pp. 423-430. 
Simmons, Robert F., "Semantic Networks: Their Computation and Use for 
Understanding English Sentences:" In Computer Simulation of 
Co~itivemProeesses, edited b$ Schanlc. R. and-Colby, K., 
Prentice Hall, N.Y. 1973, 
Walker, Donald Em, "Automat'ed Language Processing," in Annual Review 
of Information, Science and,Technolop;y Yol, 8, American Society 
Information Scien~e, Waehington, D,C. 1973. 
Winograd, Terry, "Frame Representations and the ~eclarative/?rocedural 
Controversy. " In Bobrow and ~o'lline. 
Winograd, Terry, Underatanding Natural Language. New Yotk: Academic 
Prehs, 1972. 
Woods, w.A., Kaplan, R.A., & Naeh-Webber, B.,  he Lunar Sciences 
Natural &nguage Information System: Final Report: BBN. Report 
# 2378, June, 1972, Bolt, Beranek 8 Newman, Inc. 
Cambridge, Gss. 
11 
Woods, h. A,, 
Transition Network Grammars for Natural Language 
Analysis," Cotmn. ACM, 13, Oct. 1970. 
