STEREOTYPES 
as an ACTOR Approach Towards Solving 
the .Problem. of Procedural Attachment 
in FRAME Theories.. 
Carl Hewitt 
Abstrapt 
This paper is a spin-off of our work on actors. • We have 
worked out a dictionary for translating between what M.insky 'et. ah 
are saying about frames and what we are saying about actors. Using 
PT.ASIV\[A \[PLANNER-like System iV\[odeled on Actors\].we 
can demonstrate important rela.tionships between the Minsky-frames 
and the PLANNER-like form~.lisms. PLASMA does not .use the 
Q A-4 context mechanism (Rulifson et. al. 19'12): instead It uses 
explicit tags in assertions to keep track of the state of affairs m 
various situations. One problem with the.QA-'i context mechanism 
is that the problem solver is forced to attempt to propagate all 
changes in the situation immediately on a frame, shi(t since otherwise 
inconsistent information will be inherited from the previoussituation. 
Another problem with Q A-4 context meclhanism is that it is 
sometimes difficult to reason explicitly about various ~ituations using 
it because situations \[frames\] are not explicitly part of the assertions 
and goals. Events that are Viewed from several:different viewpoints 
\[as in a murder mystery\] are diffic01t to handle. Also It Is difficult to 
retrieve the appropriate prior sltuat!ons from memory to aid In 
recognition tasks using Q A-4 contexts. However, without the 
example of the Q A-,i contexts to guide us, we could never have 
realized holy to deal with these, problems using' tagged assertions. 
The context mechanism in' CONNIVER was modeled on the one i~n 
o_..A-4. 
Actors make a contribution to the "dec\[aratlve-procedure" 
controversy in that they subsume both the behavior of pure 
procedures \[functions\] and pore declaratives \[data structures\] as 
special cases. In this paper we use actors to investigate the question 
of how to do procedural attachment to frames \[McCarthy 1969, 
• Minsky et. al. 1974\]. Actors. provide an approach to solving to the 
problem of how to attach procedures to frames in such a way thai the 
appropriate procedure is invoked with the inherited knowledge of the 
frame within Which the procedure is incorporated, as well as newly 
introduced knowledge from other frames. Our approach also 
incorporates the insight gained from PLANNER-like formalisms 
\[Hewitt 1969, 1971; Rulifson 1972; .Davies: 197'2; Sussman and 
McDermott 1972, Hewitt et. al. 19"/3\] for the'procedural embedding of 
knowledge. We introduce stereotypes as an actor version of a 
frame theory. A stereotype consists of the following parts: 
a collection of eharaoterist;io objects 
characteristic x:elations for those objects 
plans invoked bY world directed invocation 
for transforming the objects and relations 
qa 
Overview 
/\]pologyt. The schemes proposed herein are 
incomplete in many respects. First, I often propose 
representations without speci\[yi.g the. processes that will 
use them. Sometimes I only describe properties the 
structures should exhibit. \[ talk about markers and 
assignments as though it were obvious how they are 
attached and linked; it is not, " 
Marvin Minsky i~74 ~ 
FRAMES like ACTORS are difficult to define and motivate. 
Through the dint of much effort \[C,reif and Hewitt 1975\] actors are 
now becoming quite well defined \[even axiomatized!\].. In this paper 
we make a stab at similarly starting to pin down and unify various 
notions of "frames". We have observed a tendency for people to get 
hung up on extraneous details as they attempt to understand these 
difficult concepts. The•notion s of default values and expectations in 
particUlar are a cause of diff.iculty. For example expectation • is often 
confused with scientific' prediction. In attempting to sort all this out, 
we have found the methodology of CONJECTURES and 
REFUTATIONS as expounded by Karl Popper tobe of great value. 
The relationships between the assertional frames of McCarthy 
and the object frames of Minsky are explored m this paper using 
PLASMA. PLASMA allows us to demonstrate a kind or 
isomorphism between assertional frames and object frames. Using 
PLASMA accomplishes procedural attachment for both kinds of 
frames and shows how to move between object and as~ertional 
frames. 
Assertional Frames anc~ Semantic Nets. 
The example given •below was originally used by Terry 
Winograd to illustrate the properties o( Semantic Nets in his. 
informal survey entitled "Fiv.e Lectures on Artificial Intelligence'. 
Scott Fahlman is currently considering how to build a special purpose 
machine to solve such problems. We first give a formulation in 
terms of assertional frames. The addition of assertions of the form 
(... never-are ...) was inspired by Grossman \[1975\]. It is interesting to 
speculate whether the formulation in terms of assertiona\[ frames is as 
plausible a psychological model for humans as the formulation in 
terms of semantic nets. 
Formulated in Assertlonal Frames 
(Kazuo ~ people) 
((Kazuo owns Fido) in s O) 
(Fido (dogs) 
(dogs eat meat) 
(cows give meat) 
(dogs never-are people) 
• (cows nether-are people) 
(dogs never-ar~ cows) 
(people c animals) 
(dogs ¢ animals) 
(cows c animals) 
;Kazuo is a person 
;Kazuo owns Fide in situation s 0 
;Fido is a dog 
;dogs eat meat 
.;cows give meat 
;/~opte are animals 
;dogs are animals 
;eowt are anlmols 
\[to-demonstr ate-animal-eats-sor t "of'f°°d " 
;define a proceduro to show that an animal eats a sort of .food 
(¢o (demonstrate (=animal eats ,sort-of-food)) eonsider-tryint\[ 
(demonstrate (animal (=kind-of-animal) 
(thon: 
(demonstrate (kind-of-animid oat sort-of-food)))))\] 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
Formulated in Semantic Nets 
Next we formulate the example in semantic networks. 
f 
 ,oD . 
• Relationship of Worlds. to Contexts 
We conceive of worlds {Hewitt el. el. 1973} as actors that 
organize a body -of knowledge for efficient use. Worlds are 
generalizations of the tree-structured contexts developed for .QA-4 
(Rulifson el. el. 19/I) as a generalization of the data bates of 
PLANNER-69. • . 
"/J Frame is a coitection o\[ questions to be asked about a 
hypothetical situation; it apeci\[ies issue, to be ralsedand 
methods to be used in dealing with them." 
"In \[oct we shall" con~ider the idea that the .frame 
terminal, \[}.or a scenario str~ctureJ ore exactly those 
questions \[commonly associated wilh it\[."' 
Marvin Mlnsky 1974 
Worlds have many of the behaviors attibuted to 0bject frames 
{Minsky 19"H} and assertional frames {McCarthy and Hayes 1969}. In 
particular worlds exhibit a very flexible form of inheritance of 
attributes which is realized through the mechanism of message 
passing. 
Progressive Refinement of Plans 
Part of Minsky's birthday party frame includes a 
PLANNER-style general plan for getting ready for a birthday pal"ty. 
It illustrates how PLANNER-style plans.are a natural component of 
frame systems. In PLASMA the general plan can be expressed as 
follows: 
qs" 
\[echieve-guest-attends-birthday-party-for-guest-of-honor 
(to (achieve (=guest attends birthday-party \[or =guest-of-honor)) 
(choose 
(-present is-~uitobie-present~\[or guest-of-honor groin guest) 
(then: 
(achieve 
(^ 
(guest has present) 
(guest i,t,-dressed-\[or birthday-party)) 
(then: 
(achieve 
(luest is-pre~en't-rtt birtllday-porty 
\]'or guestoof-honor)))))))\] 
We use the above general plan as a template to write more 
particular plans tailored to special circumstance by progressive 
refinement of plans \[Hewitt IJCAI-?I\] of'the plan. This technique is 
an incremental glorified version of iqline substitution of procedure 
bodies for their invocations which has been ge.neralized to deal with 
pattern directed invocation. Cheatham and .Wegbreit \[19"/3\] and 
Burstall and Darlington \[1975\] have studied the problems involved in 
the absence of pattern directed invocation. 
We will suppose that early one morning that Marvin has been 
told that there will be a birthday party for Seymour the next evening. 
The above plan becomes specialized as follows: 
\[achieve-Marvin-attends-birthday-party-for-Seymour 
(to 
(achieve (Marvin attends birthday-party \[or Seymour)) 
(choose 
(=present is-suitable-present-\[or Seymour .from Marvin) 
(then: 
(achieve 
(^ 
(Marvin has present) 
(Mary n is-dressed-got birthday-party)) 
(then: 
(achieve 
(Marvin is-present-at birthday-.party 
}'or Seymour)))))))\]' 
As Marvin rehearses• his planned' actions for the day his plan 
• becomes further specialized. Eva, who is a friend of Seymour, 
probably will have suggestions as to what might be a suitable present 
for Seymour. Also, a party for Seymour Is not the kind of affair that 
guests dress up for. These considerations result in the following 
refinement of the plan: .. 
\[Marvin-attends-birthdsy-psrty-for-Seymour • 
(ask (Eva which =present is-a-suitoble-present-}'or Seymour . 
from Marvin) 
(then: 
(Marvln~ets-money '(su\[\[icient-to-buy present) 
(then: 
(Marvin-buys present 
(then: 
(Marvin-goes-tO (home-of Seymour) , 
at (time-of 
(birthd,ly-parly-ot Seymour)))))))))\] 
The above plans were formulated between yawns as Marvin 
awakened the morning, of the party 
Procedural Attachment in Assertional Frames 
"/! situation s is the complete state of the universe at an 
instant of time. We denote by Sit the set of all situations. 
Since the universe is too large for complete description, we shall 
never completely describe a situation; we shall only give facts 
about situations. These facts will be used to deduce further 
facts about that situation, about future situations and about 
situations that persons can bring about from that situation, 
• ..o 
~Ve shall further assume that the lows of motion determine 
from a situation all future "situations. {This assumption is 
difficult to square with quantum mechanics, and relat~ely tells 
us that any assignment of simultaneity tO events in different 
places is arbitrary. However, we are proceeding on tEe'basis teal 
modern pkysics is irrelevant to common tents deciding WEar to 
do, and in particular is irrelevant to solving tee 'free .will 
problem':}" 
John McCarthy 1969 
In addition to making the assumptions listed above McCarthy 
and Hayes also assume the existence of a functional which maps a 
global state in .to the "nexC global state. Although.we shall use tags 
on assertions and goals to relativize them to particular situations, 
hypotheses, and contexts we do not want to make any of the global 
assumptions of McCarthy and Hayes. All of our situations will be 
local and completely consistent with relativity and. quantum 
mechanics. For a discussion of the issues involved in this decisiott 
for the actor model of computation see Greif and Hewitt \[19'/5\]. 
The Pattern Directed Invocation \[Hewltt 1969\] incorporated in 
PLANNER-like systems accomplishes proced~iral attachment for 
assertional frames \[McCarthy 1969\]. 'This procedural attachment is 
carried down to the level of the quantificatlonal calculus •where we 
have shown \[Hewitt 1975a\] how' the logical operators of the 
quantificational calculus {V, 3, implies, ^, v, -, etc.} can be 
behaviorally defined as actors. We use the following kinds of tags in 
assertions to relatlvlze the assertions in the desired manner:. : 
• physical states 
mental states 
logical hypotheticals 
hypotheses 
view points 
goal states 
predictions 
defaults 
Stereotypes 
We introduce stereoty'pes as an actor version of a frame 
theory. Our notion' of a stereotype incorporates ideas from 
assertional frames \[McCarthy 1969\], act~ worlds \[He'witt 1969, 1973\], 
world-directed invocation \[Hewitt 1969; 1971, 19"/3; Stansfield 1975\]; 
social frames \[Goffman 197st\], and object frames \[Minsky et. al. 19'/'t\]. 
A stereotype consists of a set of the following parts: 
a collection of characteristic objects 
characteristic relations for those objects 
a set of plans invoked by world directed invocation 
for transforming the objects and relations 
%• 
It seems that many of the behaviors attributed to frames by 
Minsky can be realized by stereotypes. The characteristic objects of 
a stereotype correspond closely to the slots of a. Minsky frame and the 
characteristic relations of a stereotype correspond to the constraints 
of a Minsky frame. Minsky calls simp\]e ,ma.ry characteristic relations 
markers. We instantiate stereotypes somewhat differently from the 
way in which Minsk? instantiates object frames. Our approach is to 
plug in definite candidates for all the characteristic objects of a 
stereotype that we can and use anonymous objects for the rest. 
Defaults are done as assertions tagged tO indicate that they are 
defaulted so that they can be easily displaced if an anomaly develops. 
Stereotypes communicate by making assertions in the data base and 
by world directed invocation which is a generalization of pattern 
directed •invocation in which the invocation is done on the basis of a 
fragment Of a micro-world instead of a single assertion. Inheritance 
of attributes is done using the message passing of actor semantics. 
If a questions cannot be answered directly then parts of the job.are 
" delegated. 
Prooedur~'d Attachment for Stereotypes 
• ~ We have found it useful to incorporate ~ types of worlds 
\[called UTO\]PIA and REAI".ITY\] in .every problem solving 
situation in order to incorporate goal " orientation. The 
UTOPIA-REALITY machinery also enables us to incorporate 
PLANNING into problem solving using "islands" \[Minsky 1963\] 
as stepping Stones. The utopia of one problem solving situation is 
taken as the reality of another. 
"It will be worth a r~latively enormous effort.to find suck 
'islands' in tee solution of eomplez problems. Note teat 
even i/ one encountered, say, 106 failures o/ such 
procedures before success, one would still, have gained a 
factor of perhaps I0 tO in over-all trial reduction! T/uLS 
practically any ability at all to 'plan/ or 'anaiy~e,' a 
problem will be pro/liable r if the problem is dif\[ieul|." 
Marvin Minsky 1963 
Nested Continuation Control. Structure 
Instead of CONNIVER-style Possibility Lists 
Simple retrieval can be done using fragments of micro-worlds that 
consist of single patterns \[pattern-directed retrleval\]{Hewltt 1969}. For 
example 
(find (=apt is-an-apartment-i*n Cambridge) 
;find an al~rtment in Cambrid~re 
(then" ;then 
(refute (apt is-nn-acceptoble-apartmenr) 
;find something wren I with tee proposed apartmen! 
(else: ;else 
(move-irdo apt)i)) ;move into tee apartment 
(else: ;~Ite 
(move-to Arlington)}} ;move to flrlinKton 
The use of "then:" and "else:" continuations seemsto solve the scoping 
control problem which had been plaguing PLANNER-like languages 
for some time. CONNIVER attempted to solving the scoping control ' 
problem by introducing possibility lists and Landln-style 
non-hierarchical gotos. However possibility lists proved to •have 
several deficiencies• They introduced side-effects into the. basic 
communication mechanisms in CONNIVER which, made it difficult 
for users to debug their programs since doing a try-next operation to 
I 
I 
I 
I 
I 
I 
I 
I 
l 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
print the next possibility destructively interferes with the operation of 
the programs being debugged. The other, basic communicatioll 
mechanisms of CONNIVER similarly have intrinsic'~ide-effects built 
into their very structure. 
"Their \[Sussman and McDermott\] sohttion, to give the 
user access to the implementation primitives of 
PI,/INNER, is however, something of a retrograde.step 
(what are CONNIVER's semantics?), although 
pragmatically useful and important in the short term, t} 
better solution is to give the user access to a meaningful 
set of primitive control abilities in a~t explicit 
• representational scheme concerned with deductive cqntroi." 
Pat llayes 1'974 
Nested continuation control structure gives us the ability to 
influence or control any decision to the extent we desire. For example, 
in case of the apartment finder above, we can explicitly communicate 
complaints as to why a particular apartment is unacceptable in order 
to try to infiuence the selection of further proposed apartments. 
\[unacceptable-apartment-if-too-expensive B 
(to 
(refute (rapt is-an-acceptable-apartment) 
(with-c.omplaint-dept: =the-comptain-dept)) ' 
\[if (\[rent apt) ~; S300) 
(then: 
(the-complaint-dept <= (rent-of apt is-too-high)))))\] 
\[unaccept able-apartment -if-doesn't-have-dishwasher • 
(to 
\[refute (taps is-an-neeeptoble-ttpnrtment) 
(wlth-complalnt-dept: ,the-complain-dept)) 
(if (apt has-a dishwasher) 
(else: 
(the-complaint-dept <- (apt doesn't-have-a dishwasher)))))\] . 
Use of nested continuation control structure enables us to have the 
ability to control all o.f the decisio.ns made while still retaining the 
high level goat oriented, nature:of PLANNER. PLASMA is able to 
accomplish this by basing its semantics on actor message passing and 
slightly changing the syntax ofPLANNER-?I, The change in syntax 
provides us with natural places to incorporate the (:ontrol information 
and enables us to avoid the gratuitous side effects !n PLANNER-?L 
Unification of Pattern-Directed Invocation 
We do not want to have .to explicitly •store every piece of 
knowledge .which we have but would like to be able to derive 
conclusions from what is already• known using procedures. Using the 
distinguished symbol when with the syntax 
(when trigger consider-trying b.ody) 
or completely equivalently using the distinguished symbol to with the 
syntax 
(to trigger consider-trylng body) 
creates a plan \[high level goal-oriented procedure\] that can be 
invoked by pattern directed invocation by a trigger Which matches 
trigger. The following are all special case plans which are defined in 
terms of the above general pattern directed invocation machinery. 
(to (demonstrate hypothesis) consider-trying body) 
(when (assert statement.) cons(clef-trying body) 
(to (dlsrzchiew condition) consider-trying bo.dy) ' 
(when (deny statement) consider-trying body) 
(to (refute hypothesis) consider-trying body) 
(to (find description) consider-tr~ing body) 
(to (achieve condition) consider-tryhtg body) 
Additional kinds of plans can be defined as they are needed. 
The process of. invoking plans in worlds is controlled by 
recommendations made to the world when the plan is put in the 
world and by recommendations made at the site where a request is 
made of the world to achieve a particular goal. We envisage that 
problem solving would begin in a world with an initial class of plans. 
In many cases most of the plans that are used in. the ultimate solution 
of the problem need to be constructed by other plans during the 
problem solving process. This is ilTustrated in a limited way by the 
domain of logic which is given as an example in Hew(st \[IJCA\]-75\]. 
Pattern Directed Invocation Using AnonymousObjects 
One important way in which plans can communicate is 
through making assertions, erasures, denials using worlds which they 
share in common• Another important means of communication is 
through pattern directed invocation. An important technical problem 
in implementing pattern directed invocation is how to solve the 
problem of matching the invoking pattern with the pattern in the 
trigger of the •pattern of the plan being invokecl..PLASMA uses 
anonymous objects to solve the .problem. We assume the existence of 
a generator capable of generating new anonymous individuals anon I, 
anon 2, etc. which have never before been encountered. To show the 
utility of such a generator consider the problem of proving (x c_. z) \[x 
is a subset of z\] where we have a world which Contains: 
(x c:w) 
(x c_ y) 
(y ~ z) 
\[demonstrate-¢-= ;define the plan'demonstrate-c_ to be 
(to (demonstrate (=a~ =c)) try 
;to demonstrQte that a is a subset o'\[ e try 
(demonst#ate (a C- fib) 
;to demonstrate ihata is a subset of another set \[eall'~t b/ 
(then: ;then 
(demonstrate (b C- C) 
• ;demonstrtite t.hat b is a subset of c 
(using: set-theory))) 
(using: set-theory)))\] " 
The problem is solved by "wishful thinking'• i.e. reasoning 
within a hypothetical world. Unfortunatelywe do not have the fact 
(x ¢ z) explicitly given to us and so must do .some computation. We 
note that we have a plan whose trigger (=a c: --c) matches what we 
are trying to achieve and so turn control over to it to see what it can 
do. In order to find b_ such that (x C- b_) we let b_ be anon I which is a 
never before encountered individual which we wish to have certain 
properties. Then we note that anon I might be w. But we are unable 
to demonstrate \[w ¢__ z) so we reconsider and see that anonl might be 
y. We successfully demonstrate• (y c_ z) and so the problem is solved.. 
~7 
• A,Line Stereot3~pe 
Our first example of" a stereotype is a Per i simple one. It is an 
ordinary line. A line-segment has three characteristic objects: one 
edge and two vertices. 
Ve~rs/¢ a ~e~r~x~, 
A Line•Segment" 
(vertex s is-a-ver#e~:-of edge s) 
• (vertexb is-n-t,erter-of.edge a) 
(edge s is-the-edge-of the-line) 
There is not much that can be done with a line seg6~ent. However 
given one vertex, we of then need to be able • to get the oppos!te 
vertex. A plan for doing this is given below: 
grind-opposite-vertex 
(to (j'ind =v i is-opposile sv 2 alontr me) 
(find 
(^ 
(v I is-a-verte~:-of e) 
(vl # v2))\] 
Face. Stereotypes 
"/'he following are two common stereotypes of faces of b|ocks: 
I I 
l~eotangle Triangle 
Two Common Face Stereotypes 
In the remainder of this paper, we shall frequent.ly refer back to 
the following stereotypes. 
A Triangle Stereotype 
A triangle stereotype contains ~three objects: all of which are 
lines. 
" 
I 
The Important characteristic relation in the triangle stereotype is the 
relationship of that all three lines join each other. 
(the-triangle E convex-polygons) 
i¢{a ... c} 1implies (line i is-an-line-of the~lriangla) 
(line a joints line b) 
(line s joinls line c) 
(line b joints line b) 
A Rectangle Stereotype 
A rectangle stereotype contains four objects: all of which are 
lines. 
I 
I 
I 
I 
I 
I 
(the-rectangle (convex-polygons) • 
i({a ... d} implies (line i i*-a-ilne-o$ the-re.ctangle) 
(line e joins line b) 
(line b joi~, llne¢) 
(line c joins llne d) 
(line d joins line s ) 
(line a is-lmrailel-to line c) ' 
(l|ne b ia-parnUel-to line d) 
(line s is-opposite line c) 
(line b is-opposite line d) 
Blook Stereotypes 
The following are three common stereotypes of blocks: 
I 
I 
I 
I 
I 
Cube Wedge Square-Pyramid 
Three Common• Block Stereot~vpes I 
A Cube Stereotype . " " ' I 
J A cube stereotype contains six objects \[called faces\] which are 
squares. 
• ~.E b I 
I 
I 
I Two important characteristic relations in the cube stereotype are the 
l relationship of two faces being adjacent two each other and the 
relationship of two faces being opposite each other. 
(the-cube (convex-polyhedra) 
i~{a .., f'} imldif, s (face i i~-a-faee-of the-cube) 
i({a ... f} iml~liox (face i is-square) 
I (face a is-Ol~l)o~ile facef) ' 1 
(face s is-adjacent-to faceb) 
(face s is-adjacent-to facet) 
i (face b is-adjacent-In faCec) 
A ~vled~e Stereotype 
I A wedge stereotype contains five objects \[called faces\], three of 
which are rectangles and two of which are triangles. The two 
triangular faces are opposite each other. 
a 
I 
I 
I 
I 
I 
(the-wedlze (convex-polyhedrs) 
i({a ... e} iml)lles (face i is-a-fae~-oJ" the-wedge) 
(face n i~-tda,~ular) 
(face b is-tdan~ular) 
(face s is-opposile faceb) 
A Square-Pyramid Stereotyp- 
A square-pyramid stereotype contains six objects: four of which 
are triangular faces, one tsa square face, and one Is a vertex. 
-~e-¥egTU~< 
I 
I 
I 
(the-square-pyramid (convex-polyhedra) 
i(\[e ... e} implins (face i is-a-face-of the-squire-pyramid ) 
i({s ... d} implies (face i ia-Ifiangular) 
(face e is-th~,-baso) 
(the-vertex is-oppodlo fncee) 
Stereotyped Views of Models of Blocks 
A stereotype for a typical view. of a cube stereotype contains 
three sides, one Y vertex, three arrow vertices, and three L vertices. 
I 
I 
I 
((side s is-a-vleu~-of lace s af block i) in the-view) 
((side b is-a-view-el taceb, of block i ) in the-view) 
((side c is-a-t, lew-of face c of block 1) in the-view) . 
((side s i~-Ieft-helaw side c) in the-view) 
((side b ia-rlgEt-kelou~ side c) in the-view) 
((v e is-a-Y-retreat) in the-view) 
((v b i~-an-arrow-t,r, rte:r) in the-view) . 
((v c i~-nn-I.-vertox) i, the-view) 
((v d is-an-arrouH,erteJ:) in the-view) 
((v e is-an-l,-T~erte~c) in the-view) 
((vf is-an-nrrow-t,~,rto=\] in the-view) 
((vg is-an-l,-v~rtex) in the-view) 
Stereotype for Typical View of a Cube 
Reoognitior~ of Stereotypes 
~'e make proRresl if, and only if. w~. are prepared I0 
learn from our mistakes. 
Karl R. Popper 
Philosophically, this section builds on some work by Karl 
Popper on Conjectures and Refutations as a scientific methodology. 
More concretely it builds on a fine piece of work Ben Kuipers did on 
recognition of kinds of blocks \[Kuipers 1974\]. The reader is referred 
to KuIpers! paper for various kinds of discussion and background 
which will not be repeated here. The recognition problem Is to 
incrementally recognize the above three kinds of blocks. This kind 
of problem has been extensively investigated by.robot projects at the 
M.I.T. Artificial intelligence Laboratory, the Stanford Artificial 
Intelligence Laboratory, and the S.R.I. Robot Project. In order to 
facilitate comparison, we have fairly closely followe.d Kuiper's 
scenario. Our motivation in re-exploring the block recognition 
problem is to Investigate some techniques that have been developed 
by Marllyn McLennan for the very difficult. 'domain of 
understanding plant pictures. We wanted to investigate a technique 
that she is developing for resolving conflicts, between conflicting 
hypotheses recognition problems in another domain. 
We have modified Kulpers scenario of the recognition of a 
block using stereotypes by using three-dimension stereotypes Instead 
of the two-dimensional views which he used. These 
three-dimensional stereotypes are reminiscent of. some of the 
three-dimensional "models" used in the robot vision programs. We 
start the recognition with an initial vertex v I (see figure below), 
which in this case happens to be an L-vertex. Our initial hypothesis 
h is that the drawing represents a cube as indicated by the dotted 
lines: 
:.:'.. - ...... ,. ".-: : , 
i 
t 
t_ ...... !..."" .~ ez. 
Vertex I 
We have the following situation: 
q~ 
(v i is-a-,,ertex-of e i) 
(v I is-a-vertex-of e 2) 
(v I i~-an-L-,,erte~) 
((block| iS-~z cube) in b) 
((v| correspo,ds-to vg) in h) 
e+ "':'" I 
¢ I ! 
" • • 
~1 ~ Vertex 
We now have the following information: 
(v 2 is-an-arrow~vertex) 
(V 2 is-a-t~ertex-of ®l ) 
(V 2 is-a-tJerte~:-ol e3) 
(v 2 is-a-tJerte~:-o\[ • 4) 
((v 2 correspondx-tO Vb).in h) 
The second vertex 
hypothesis h. 
observed 
V~ "= .... ~.-'"" t 
l . l~"e, 
Vertex 3 
fits in completely with the 
{v 3 is-nn-arrow-~ertex) 
(v 3 is-a-Uertex-a\[ • 2) 
• (v 3 is-a-verte~¢-ol • 5) 
(V 3 is-a-verte~-o\[ • 6 ) 
((v 3 corresponds-to v() in h) 
This vertex fits the hypothesis h of the cube stereotype, since it 
is the anticipated arrow vertex. 
v, 
. . ; °: 
! °! 
V, eL v~ 
Vertex 4 
We assume that the visual primitives that we are using are 
region-oriented as well as being line.oriented so that we can directly 
detect and recognized the shape of certain kinds of regions. 
I 
(side I is-n-parallelogram) 
(v 4 is-a-Y-,sertex) 
(v 4 is-a-vertex-of • 4) 
(v 4 is-a-vertez-of • 5) 
(V 4 is-a-vertez-of e 7) 
((v 4 corresponds-to V a) in h) 
((side I is-a-view-of face a of cube 1) i, h) 
The Y-vertex at the center of the figure also corresponds 
completely with the cube hypothesis h. A complete parallelogram face 
has now been observed and confirmed. 
v, "'" ,, .s;.,,. 
• ~C. '• 
~Sto~ t 
~| • ' ' 
• ' Vertex 5 
(side 2 is-a-triangle) 
(v 5 is-a-arrow-vertex) 
(v§ is-a-vertex-of %) 
{v 5 is-a-~ertex-o\[ e 7) 
Confrontation and l~efutation 
I 
I 
I 
I 
I 
I 
I 
I 
With this observation, the cube hypothesis h finally breaks i 
down. The anomaly occurs because the assertion that s.ide 2 is a II 
triangle conflicts with the characteristic relationof the cube s~ereotype 
that all of its faces are squares. Thus there Is .no easy to resolve the 
anomaly.by simply rotating the cube• However, the cube stereotype i 
is reluctant to give up completely so it Imagines one of its sides II 
shrunk down to an edge to fit the data. This reminds cube stereotype 
of the the wedge sterfotype, which it suggests. The triangular side, in 
effect, caused the recognition system to do a "double-take" creating i 
hypothes'is h' that the block isa wedge since the wed|e stereotype Is I 
happy with the situation. 
(h' obtained-by-refutation-of h) I 
((block I is-a wedge) in b') | 
((side I is-a-view-of face c ofwedge 1) inh') 
((side 2 is-a-dew-of fKo a of wedge 1) i~i h') 
At this point, with. the wedge stereotype directing the 
exploration, there is only one remaining edge. Unfortunately it. 
refutes the hypothesis h' 
U/" I V, ec.-,~ 
V~ 
Vertex 5 ConUnued 
(v 5 i,-a-~rtez-of • 3) 
(side 3 im-a-trlangle) 
The observation that side 3 is a triangle conflicts with the 
I 
I 
I00 I 
characteristic relation of the wedge stereotypetheir a wedge does not 
have two triangular faces Which are adjacent. Again no amount of 
rotation will help at all. However the triangular face side 3 suggests 
squeezing the edge of the wedge down to a point suggesting a 
pyramid. The pyramid stereotype takes a look at side I and decides 
that the square-pyramid stereotype should be invoked, 
(h" ohtnlnr, d-hy-refulatinn-o/" h') 
((block i is-tz square-pyramid) in h") 
((side I is-n-i~i~w-o\[ face eo\[ square-pyramid 1) in h") 
((side?. is-a-t;iew-o\[ face a af square-pyramid\[) in h") 
((side 3 is-~z-f,iew-o\[ face b o J" squareopyramid I ) in h"). 
Since there is no further input data to be considered and .thus 
further processing yields no.refutations, hypothesis h' temporarily 
survives. 
T_racking the Image of a, Cube 
In this section we shall consider ,an example due to Mlnsky 
from the point of view of stereotypes. We assume.that the result of 
looking at the cube pictured below 
I 
Of course the situation of the cube has not changed but we have the 
following additional information: 
((C has-color white) in situation 1) 
(C is-a-view-o j" facef o\[ cube l) . . 
How if we imagine moving back to \[he left, we can hypothesize the 
view without any perceptual computation at all. We simply imagine 
that the new view view 3 is the same as view i. 
((view 3 = view 1) in hypothesis l) 
However, moving back to the left we are surprised to find that 
in the new view \[view 3\] that A has changed color to while! Thus 
hypothesis ! is rejected. We also notice that 
((floor has flecks-of-white-paint) in view3)" 
which causes us to construct another hypothesis" 
(situation 2 = (pain! A white in situationl)) 
((view 3 is-a-IJiew-o\[ situation 2) in hYpothesis2) . 
Further careful observation and testing.does not refute 
hypothesis 2, so situation 2 can inherit suitably transformed attributes 
from situation I using the hypothesized paint transition. 
More .°n Inheritan,ce ofAttributes 
The "frame" problem of McCarthy for assertional frames 
corresponds closely to the problems of "inheritance of attributes" and view I 
I " "default values" for object frames. For example the fable quoted by 
is the following symbolic description: Minsky about the wolf and the lamb is very close to the frame 
problems of McCarthy. 
((A has-color red) in view I ) 
I ((B has-color while)in view|) . .Recently Keith Nishihara has reconsidered the problem of 
((E hr=s-eelor white)in view I ) gluing blocks together from t.he point of inheritance of.attributes. 
((A is-lej't-below E)in'view 1) " Suppose that there actually we:re two cubes In the first situation 
((B is-riEht-Eelow E) in view i) . " • considered above 
I ((A has-ahtlpe-n\]" verticll-p|rellelogram).in view I ) 
((B h,',,-~kepe-oj" vortical-parallelogram) in vi';w I ) 
I The above view represents the situation below: 
(view i ie-a-view-e\[ situation I ) 
(A is-a-view-o\[ face a o\[ cube 1) 
I (B is-a-vlew-o\[ face bo\[ cube I ) view 4 of situation! 
• (E in-a-view-of face s of cube 1) so we have the following additional assertions: 
When we move to the right, face A disappears fromview, 
I while the new face decorated with C is now seen. 
I 
I view 2 
I The new view has the following symbolic description: (view 2 • (move-to-right-around cube I from viewl )) ((C has-color white) in view I ) 
(view 4 is-t~-vlew-of situation I i 
(populationsituationl blocks) ,, 2 
(\[A' has-color red) in situation!) 
((B' he J-color while) in situation 1) 
• ((E' has-color white) • in situation l) 
(A' |a-a-riP.w-o\[ face a o j" cube 2) 
(B' is-a-view-of face bo\[ cube2 )~ 
(E' ia-a-vlew-o\[ facet o j" cube 2) 
Now we can create situation 4 by gluing side C of cube I to side A' of 
cube 2 to give block I. 
I 
(populationsituation4 blocks) = i 
(situation 4 re.~ults-frorn (glue C to A ~, in situation l)) 
((block I (blocks) in situations,) ' 
((ma s block I ) = ((mass cube ! ) ÷ (mass cube2))) 
(((face a o,r cube i ) i.~-a-/ace.-o\[ block/) in situation4) 
(((face a o,r cube i ) he~-coior red) in situation4) 
Note that some attributes in siluation 4 such as the color of A are 
inherited directly from one of the cubes in situationl: ' On the other 
hand the mass of block I is inherited as the sum.of the masses of 
cube I and cube 2. 
Real World l~eoognition 
The principal reason why vision programs at present perform 
so poorly is that the amount of knowledge they can bring to bear on 
the seeing process is so limited. For example, Waltz's program \[Waltz 
1972\] is the latest in a line of development called scene analys, which 
was originated by Guzman \[1968\], and pursued by Huff man \[1970\] 
and Clowes \[1971\]. The usefulness' of this appro;ich is called into 
question by the difficulty of extracting, from information about 
intensity and color, the near perfect line drawing that such programs 
require; and by the restricted nature of the line drawing 
represeniation itself~ Waltz has demonstrated" that when a more 
detailed categorization of the kinds of labels is made that the number 
of ambiguous line drawings decreases dramatically. " 
We greatly admire Waltz's program, .but we f~l that the 
apprOach that it embodies is open to several criticisms. The first is 
that the knowledge that it uses is in a certain sense not explicit 
enough. Although it contains a great deal of. information about the 
appearance of line drawings, this information is essentially, in a 
compiled form: one reflection of this ts that the structure of Waitz's 
program makes it inherently unable to use either explicit information 
about the three-dimensional form of what is being viewed, orthe 
many pieces of special and general knowledge that we surely bring to 
bear on the process of seeing.' Essentially, the only way one.can 
attempt to influence the program is by adding or deleting junctions 
from a large table of "legal" labellings. There is no way in which 
pieces of its knowledge can be pulled out and examined while tt tries 
to create an interpretation of, for example, a scene in which several 
lines are missing. Unless such knowledge, suitably embedded in a 
hypothetico-deductive system, can play a large part in the operation 
ofa vision program, we see no prospect of such a program being 
able to interpret the incomplete information that is the diet of daily 
life. 
The basic trouble with the labelling approach of scene analysis is 
that it is too limiting and stultifying a paradigm for vision, in much. 
the same way that resolution is for deduction. The fundamental 
• principle of resolution, that (- A) and {A v B,) together imply B, is 
occasionally useful, But attempting to make a uniform resolution 
proof procedure, to mechanize deduction in a way that cannot be 
very sensitive to hints, hunches, and a wide variety or higher level 
knowledge about the particular domain in question, is a cul-de-sac. 
Similarly, the line and vertex labels are local, predicates that are 
occasionally useful, and are of •some mathematical interest in their 
own right: but the problem of creating a uniform procedure to label 
arbitrary line drawing is not a central one for vision. Hence, we 
believe that the kind of knowledge contained in Waltz's program is 
probably relatively unimportant; and that the way in which it is made 
available there is certainly too restricting. 
The proper endeavor of vision research is to decide what 
k.nowledge should be used to help a vision system to see, and to 
discover methods that make it possible to use such knowledge. How 
can one pursue this goal more effectively? There are several kinds of 
answers. The first is to abandon the restrictive format of line 
drawings, so that programs can use information about visual features 
that are not coded in this form. There is a large gain to be had by 
loosening up our formalisms to incorporate the many .pieces of special 
and general knowledge that are necessary for purposeful vision in the 
real world. Vision is primarily a.util tarian functmn in order to see 
properly it is necessary to know what kind of information is sought. 
We believe that progress in particular domains of recognition 
on the following problems will pay. large dividends: 
Getting Started: How to proceed from a set of 
feature clues to more global hypotheses. How to 
incorporate parallelism into this process: 
Keeping Going: How to recognize and mechanize 
confrontations between cohflicting hypotheses~ How to 
retain and use valid information that was incorporated in. 
hypotheses that have been refuted. 
Furthe~ .Work 
The PLASMA system described in this paper is currently being 
implemented at the MIT Artificial lntell!gence Laboratory. Thebest 
version which currently runs was coded by Howle.Shrobe. A better' 
humanly engineered version has been designed and coded by Carl 
Hewitt In PLASMA with the extensive aid of Marilyn McLennan, 
At the time of this writing \[March. 1975\] the new implementation is 
being translated into LISP by the MIT laboratory course 6.89'3: 
"Implementation and Application of Actor Systems'. A rough draft 
of a primer by Brian Smith and Carl Hewittfor the new 
implementation exists. 
A._ ck .nowledgements 
The research reported in this.paper was sponsored by the MIT 
Artificial Intelligence and Laboratory and Project MAC under the 
sponsorship of the Office of Naval Research. 
The" main example in this paper is•adapted from a very niece. 
piece of work by Ben Kulpers. The major change we made in his 
scenario was to introduce three-dimensional models and to try to 
reason as much as possible in three-dimenslonaJ terms. Marilyn 
McLennan, Ben Kuipers, and Candy Bullwinkle made helpful 
comments and criticisms which considerably improved the 
intelligibility and content of this paper. The progress we havemade 
on actors would have been completely impossible without the 
contributions and questions of numerous MIT students• Ben 
Kulpers, Howie Shrobe, Keith Nishihara, Brian Smith, Akl 
Yonezawa, Richard Steiger, and Peter Bishop, and Irene Greif have 
done much of the work in making actors intelligible and relevant to 
the problems of constructing knowledge-based systems. Irene Grelf is 
doing exciting work in extending behavioral semantics to the area of 
inter-process communication, and puttingsemantics on a more social 
foundation• Ho.wie Shrobe, Brian Smith, Todd Matsun,. Roger Hale, 
and Peter Bishop have contributed to the multitude of "throw-away" 
implementations through which •have we debugged many. Of our 
ideas. 
I 
I 
l 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
l 
I 
I 
I 
I 
I 
I 
This paper builds on ex{ensive previous work on frames, 
paradigms, and hypotheses done b.y Fahlman, Hewitt, Kuhn, Kuipers, 
McCarthy, Minsky, Nishihara, Polya, Popper, and Winograd. Our 
notion ot ~ a stereotype is strongly reminiscent of the the logical notion 
of a formal system. However, stereotypes are used quite differently. 
Putting tags on assertions as advocated in this .paper to record the 
state of affairs in various' local situations seems more powerful and 
flexible than the context mechanism invented by Rulifson for Q A-~ 
and later adopted by CONNIVER. Tags .on assertions have 
previously been used to a limited extent in PLANNER-69 
\[MICRO-PLANNER\] by Charniak and Biss to record situations. 
They are closely related to the global ~ituations \[frames\]of McCarthy. 
The criticism of the line-labelling approach to vision comes from 
Marr and Hewitt 1973. Many of the ideas have emerged from the 
M IT course 6.893: "implementation and Application of Actor Systems" 
given in the spring of 1975 with the following participants: Russ 
Atkinson, Mike Freiling, Kenneth Kahn, Keith Nlshihara,Marilyn 
McLennan, Howie Shrobe, Kathy Van Sant, and Aki Yonezawa. 

Bibliography 
Biss, K.; Chien~ R.; Stahl, F.; and Weissman, S. "Semantic Modeling 
for Deductive Question-Answering" Proceedings of. 
IJCAI-73. •August \[973. Stanford. pp. 356-363. 
Burstall, R. M. and Darlington, j. "Some T ran'fformations for 
Developing Recursive Programs" International Conference 
on Reliable Software. Los Angeles. April,'lg"lS. 
Cheatham, T. and B. Wegbreit, "A Laboratory for the Study of 
Automating Programming" SIGSAM Bulletin, Vol. 21, 
• January, 1972. 
Davies, D. J. M. "POPLER: A POP-2 PLANNER'.MIP-89. School 
of A-I. University of Edinburgh. 
Davies, D. J. M. "Representing Negation in a PLANNER System ~ 
Proceedings of AISB Conference• july, 1974. University 
of Sussex. 
Greif, I. and Hewitt, C. "Actor Semantics of .PLANNER-73" 
Proceedings of ACM SIGPLAN-SIGACT Conference. 
Palo Alto, California. January, 1975. 
Grossman, R. W. "Representing theSemantics of Natural Language 
as Constraint Expressions" MIT Artifl<:|al Intelligence 
Working Paper 87. January, 1975. 
Hayes, P. J. "Some .Problems and Non-Problems !n Representation 
Theory" Proceedings of AISB. Sussex July, 1974~ 
Hewitt, C. "PLANNER: A Language for Manipulating Models and 
Proving Theorems in a Rqbot" IJCAI-69, Washington, D• 
C. May 1969~ .. 
Hewitt, C. "Procedural Embedding of Knowledge in PLANNER" 
IJCAI-71. London. Sept, 1971• 
Hewttt, C. "Procedural Semantics" in Naiural Language Processing 
Courant Compu'ter Science Symposium 8. Randall Rustln, 
editor. Algorithmics Press. 1971. 
Hewltt, Carl et..al. "Actor Induction and Meta-evaluation" 
Conference Record of ACM Symposium on Principles of 
Programming Languages. Boston. Oct, 1975. " 
McCarthy, J. and P. Hayes, "SomePhilosophical Problems From the 
Standpoint of Artificial Intelligence" Machine lntel!igence 
Vol. 4. American Elsevier Publishing .Company. New 
York. 1969. 
McDermott, D. "Assimilation of New Information by a Natural 
Language Understanding System'• MIT AI Memo 298. 
March, 1974. 
McLennan, M, "Hypothesis Formation as a Method of 
Understanding Plant Pictures" Thesis Progress Report for 
Fall 197'L Private Communication. 
Minsky, Marvin. "A Framework for Representing.Knowledge" MIT 
AI Memo 306. June 1974. 
Moore, J• and Newell, A. "How can MERLIN understand?' 
Department of Computer. Science. Carnegie-Mellon 
University. 1973. 
Popper, K. Con jectures and Refutations Basic Books. 1962.' 
Rulifson Johns F., Derksen J. A., and Waldinger R. J. "QA4: A 
Procedural Calculus for Intuitive Reasoning" Ph. D. 
Stanford. November 197~.. 
• Smith, B. C. and Hewitt, C. "Towards a Programming Apprentice" 
Proceedings of AISB. Sussex. July, 19"/4. 
Stansfield, J. L. "Programming a Dialogue Teaching Situation" 
Ph.D. Thesis, Edinburgh: University of Edinburgh• 
January, 1975. 
Waltz, D. L. "Generating Semantic Descriptions from Drawings Of 
Scenes with Shadows" M.I.T. Technical Report' TR-271. 
November, 1972. 
Winogracl, T. "Five Lectures on Artificial Intelligence" Stanford. A.I. 
Memo. September, 1974, 
Hewitt, C. "Protection and Synchronization i.n Actor Systems" M.I.T. 
working paper. November, 1974. 
Kuipers, 
Hewitt, C. "The Semantics of ACTIONS and the Semantics of 
TRUTH" IJCAI-75. Tiblisi, USSR. Sept. 3-8, 19"/5. 
Kuhn, T. The Structure of Scientific Revolutions Univ of Chicago 
Press• (2nd. Ed.) 1970. 
Ben. "An Hypothesis-Driven Recognition Sysiem for the 
Blocks World" MIT-AI Working Paper 83. March, 197't. 
Marr, D. and Hewitt, C. "VIDEO ERGO SC\[O: Anessay on some 
thing we would like a vision system to know" M.I.T.A.I. 
Lab Working Paper 60. November, 1973. 
McCarthy, J. "Programs with Common Sense" Proceedings of the 
Symposium on the Mechanization of Thought Processes. 
Teddington. HMSO. London. 
