CONCEPTUAL TAXONOMY OF JAPANESE VERBS FOR UNDERSTANDING 
NATURAL LANGUAGE AND PICTURE PATTERNS 
Naoyuki Okada 
Department of Information Science 
and Systems Engineering 
Oita University 
Oita 870-11, Japan 
Summary 
This paper presents a taxonomy of "matter 
concepts" or concepts of verbs that play roles 
of governors in understanding natural language 
and picture patterns. For this taxonomy we asso- 
ciate natural language with real world picture 
patterns and analyze the meanings common to them. 
The analysis shows that matter concepts are di- 
vided into two large classes:"simple matter con- 
cepts" and "non-simple matter concepts." Fur- 
thermore, the latter is divided into "complex 
concepts" and "derivative concepts." About 4,700 
matter concepts used in daily Japanese were ac- 
tually classified according to the analysis. As 
a result of the classification about 1,200 basic 
matter concepts which cover the concepts of real 
world matter at a minimum were obtained. This 
classification was applied to a translation of 
picture pattern sequences into natural language. 
1 Introduction 
As is generally known, the intellectual ac- 
tivities of human beings are very instructive in 
higher processing of natural language and pic- 
ture patterns, especially real world picture 
patterns. There are three sides to intellectual 
activity: 
(i) Recognition and understanding. 
(2) Thinking und inference. 
(3) Expression and (intellectual) action. 
The system of concepts or knowledge plays an 
essentially important role in each activity. The 
base of the system is considered to be placed on 
those concepts formed by direct association with 
the real world, which are closely related with 
both syntactic and semantic structures of nat- 
ural language. The aim of this paper is to make 
this system clear from the linguistic viewpoint. 
1-3 
There are two linguistic approaches to the 
analysis of the system. One is the understanding 
of the outline of the whole system and the other 
is the detailed analysis of a small part of the 
system. Compilation of a thesaurus is considered 
of the former type. Thesauruses compiled so far, 
4,5 however, are not sufficient for machine pro- 
cessing because of the following: 
i. Abstraction processes of concepts 
As shown in Sect. 2.2, it is important to 
introduce abstraction processes or conceptuali- 
zation processes to the system not only for its 
systematic analysis but also for the "under- 
standing'of natural language and picture pat- 
terns. The processes are not taken into consid- 
eration in ordinary thesauruses. 
2. Interrelation among concepts 
To know semantic interrelation among words 
are indispensable for natural language process- 
ing. This information is not explicitly express- 
ed in ordinary thesauruses. 
3. Criterion for classification 
In machine processing it must be shown why a 
word is classified into such and such term. Or- 
dinary thesauruses do not stress the criteria. 
Concepts of verbs are the core of the system 
from the linguistic viewpoint. We classify al- 
most all concepts of verbs in daily Japanese by 
association of natural language with the real 
world, answering the above-mentioned problems. 
As for problem i, a working hierarchy along an 
abstraction process is constructed in the system 
As for problem 2, case frames are shown in "sim- 
ple matter concept," and connecting relations 
among elementary matter concepts are shown in " 
non-simple matter concept." As for problem 3, an 
algorithm is introduced into the classification. 
2 Preliminary Considerations 
2.1 Meaning Common to Natural Language and Pic- 
ture Patterns 
Putting aside what the meaning of a picture 
pattern is, let's first discuss how it can be 
understood. When a picture pattern or picture 
pattern sequence is given, an infinite number of 
static or dynamic events can generally be ob- 
served within. Suppose that the meaning of each 
event is described in natural language--in fact, 
one can express almost all events in natural 
language apart from the question of efficiency__ 
these descriptive sentences will amount to an 
infinite number. An ordinary sentence is reduced 
into simple sentences, each of which is governed 
syntactically and semantically by a verb. Since 
there is a finite number of verbs in each lan- 
guage, the meanings of an infinite number of the 
events involved are roughly divided into the 
meanings of those verbs and their interrelations. 
Now, what is the meaning of picture pat- 
terns ? In the case of circuit diagrams or chem- 
cal structural formulas, we can think of the se- 
127- 
mantics because they have signs and syntactic 
relations. In the case of real world picture 
patterns, however, there exists neither signs nor 
syntactic relations. Here we observe real world 
objects named by human beings. If we consider 
them something like signs, we can think of the 
syntax, and then the semantics, too. The mean- 
ings are common to natural language and picture 
patterns, although their syntactic structures 
differ largely from each other. 
2.2 Paradigms for Interpretation and Understand- 
k~ 
In order to clarify the notions of interpre- 
tation and understanding, first, we propose a 
working hierarchy of knowledge along the ab- 
straction process, as follows: 
Level 1 Raw data Data close to copies of 
Level 2 
Level 3 
Level 4 
Level 5 
O0 
0 
0 L~ 
i 
Level 5 
things and events in the real world. 
Image-like data. 
Data of visual features Features 
extracted from raw data. 
Data of conceptual features Sym- 
bolic data associated with visual 
features. Some of them correspond to 
Chomsky's syntactic features in the 
lexicon. 6 
Concept data Data obtained by or- 
ganizing conceptual features. Most 
data have names as words. In case of 
the verb they roughly correspond to 
Minsky,s surface semantic frames. 7 
Interconnected concept data Net- 
works of concept data. A concept can 
be interconnected with other con- 
cepts from various viewpoints. 
Interconnected 
concept data \[ 
Level 4 
\[Data of conceptl 
Level 3 
Data of concep-\] 
tual features 
Level 2 
Data of visual 
I features 
003 Level 1 
H~ Raw data 
 (Level 0 ')\] 
~ ~ Lktion process\] 
Schank,s scripts can be regarded as 
one of this type. 8 Some networks 
have names as words. 
Fig. 1 shows the hierarchy. "Interpretation" 
is considered as an association of the data at 
one level with another level. (Here input images 
are considered as level zero data.) Since the 
knowledge system has several levels and each 
level has many domains, interpretation is possi- 
ble in many ways. If an interpretation is per- 
formed under a certain control system that spec- 
ifies which level and which domain the input 
data should be associated with, it is called " 
understanding." 
As the level number increases, a level be- 
comes higher because abstractions of concepts 
proceed. But, which is deeper, level 1 or level 
5 ? In natural language understanding, input 
sentences will probably be interpreted initially 
at level 4 or 5, then the interpretation may de- 
scend to level i, where level 1 might be deeper 
than either level 4 or 5. However, if the inter- 
pretation of a picture pattern proceeds from 
level 1 to 5, we think level 5 as the deeper 
level. 
The knowledge system is so massive and com- 
plicated that it is necessary to make systematic 
analyses. Since the number of verbs are finite, 
concepts of verbs at level 4 provide a clue to 
systematic and exhaustive analyses of knowledge 
from the linguistic viewpoint. 
The concepts of verbs are divided into two 
large classes:"simple matter concepts" and "non- 
simple matter concepts." 2,3 
3 Simple Matter Concepts 
A ,. with the real world e I : It has a roof. c I : Time lapse 
I e 2 : Man lives in it. c 2 : Moving from inside to 3.1 Structural Patterns .... outside. 
ds(s) : Subject is things. An object in the real world 
A @ > A 
Visual organ 
Fig. 1 Hierarchy of the knowledge system 
The simple matter concepts 
are not reduced into any more 
elementary matter concepts 
while the non-simple ones are 
reduced. Most of them are so 
concrete that they are well 
analyzed by direct association 
identified by a verb is called 
"matter." Unlike things matter 
does not occur alone. It arises 
accompanied by things, events, 
and attributes, which are call- 
ed "constituents," so this con- 
cept can be regarded as the 
concept of a dynamic or static 
relation among constituents and 
be expressed by 
v(s,o,of,ot,om, Os,Ow,Oc, 
p,t,r,. .... ) (A) 
where each symbol in parenthe ~ 
ses represents a constituent 
specified below. 
s : subjective concept 
o : objective concept 
of: starting point in ac- 
128 -- 
tion, or initial stste of change 
ot: finishing or target point in action, or 
final state of change 
Om: opponent in mutual action 
Os: standard or reference 
Ow: way or means(including instrument) 
Oc: concept which supplements attributive 
aspects 
p,t,r, ..... : place, time, cause(or reason), 
..... 
Out of these, eight constituents s through o c 
are obligatory because they are indispensable 
for the recognition of matter. In Japanese sen- 
tences, the obligatory constituents are often 
accompanied with such postpositional words as s- 
ga, o-o, of-kara, ot-ni, om-to , os-ni, ow-de, 
and oc-tO. But it is difficult to decide the 
case of a constituent only by such postposition- 
al words. 
The combination of obligatory constituents 
decides the basic frame of matter concepts. Ta- 
ble 1 was obtained after an elaborate investiga- 
tion of more ~han 1,500 simple matter concepts. 
Two comments must be added to Table i. First, 
optional constituents participate fairly free- 
ly in matter. Table 1 says nothing about this 
problem. Next, some obligatory constituents are 
not obligatory in every case. 
M1 (konoha-ga eda-kara) ochiru. 
(A leaf) falls (from the branch). 
M2 (botan-ga shatu-kara) toreru. 
(A button) comes off (the shirt). 
In M1 of eda(branch) is optional because ochiru 
is recognized by observing the vertical movement 
of a leaf, while in M2 of shatsu(shirt) is ob- 
ligatory because toreru is not recognized with- 
out the existence of a shirt. Constituents of, 
ot, Ow, and o c belong to such a group. 
3.2 Semantic Contents 
In case of semantic contents it is difficult 
to classify them by examining the combination of 
constituents, so we adopted a trial-and-error 
method extracting features for classification 
from the concepts. Letting a set of simple mat- 
ter concepts under consideration be C, the fea- 
ture extraction from £ is performed by the fol- 
lowing recursive procedure: 
Step 1 
Select several elements having similar con- 
tents from ¢ and extract from them a feature (~) 
which makes them similar. 
Step n(>2) 
Let the features extracted up to step (n-l) 
be Cl, c2,. .... ,Cn_ I. Extract a feature (c n) in 
the same way as step i. (The element so far se- 
lected may be adopted in the extraction.) And 
compare c n with each ci(l~i_<n-l). 
i) If c n is independent with each ci, adopt it 
as a feature and go to step (n+l). 
2) Otherwise, 
2.1) if the contents of Cn/C i contains that of 
ci/Cn, adopt c n as an upper/lower-grade feature 
of c i and go to step (n+l). 
2.2) Otherwise, make c n as a special feature 
and go to step (n+l). 
Table 1 
No. Pattern 
I 
Z 
IV 
V 
VI 
VII 
VIII 
IX 
X 
XI 
v(s) 
v(s,of) 
v(s,ot) 
v(s ,o m) 
v(s,os) 
v(s,o) 
v(s,o,of) 
v(s,o,ot) 
iV(S,O,O m) 
v(s,o,ow) 
v(s,o,oc) 
Types of structural patterns 
Example 
(konoha-ga) ochiru. 
(A leaf) falls. 
(otoko-ga ie-kara) deru. 
(A man) goes (out of the house). 
(tar~-ga yu~inkyoku-ni) iku. 
(Taro) goes (to the post office). 
(torakku-ga basu-to) butsukaru. 
(A truck) collides (with a bus). 
(ko-ga oya-ni) niru. 
(Children) resemble (their par- 
ents). 
(hanako-ga ringo-o) taberu. 
(Hanako) eats (an apple). 
(untensyu-ga tsumini-o kuruma- kara) orosu. 
(A driver) unloads (baggage from 
the car). 
(aeito-ga kaban-ni kyb~kasyo-o) treru. 
(Pupils) put (textbooks into 
knapsacks). 
(hikUshi-ga kanseit~-to shing~-o) kawasu. 
(A pilot) exchanges (information 
with a control tower). 
(hito-ga saji-de sate-o) suk~. 
(One) scoops (sugar with a spoon). 
(hito-ga soyokaze-o suzushiku) kanjiru. 
(Men) feel (a gentle breeze cool). 
X~ Others 
Table 2 Features of semantic contents 
No. Semantic feature Example Dis~ 
0"0 
0.i 
0"2 
0"3 
0"4 
i" O0 
i'01 
1"02 
1"03 
1"04 
1"05 
i" 06 
1"07 
1.08 
1.09 
i. i0 
2.0 
Displacement 
Change in the direc- 
tion 
Deformation A 
Spiritual change 
Sensual change 
Deformation B 
Change in quality 
Change in quantity 
Optical change 
iColour change 
iThermal change 
I Change in force and 
energy 
Vocal change 
Occurence,appearance and disappearance 
Start,end and stop 
Temporal change 
Continuation 
ochiru(fall) 
mukeru(turn) 
magaru(bend) 
okoru(get angry) 
kanjiru(feel) 
yaseru(get lean) 
kusaru(rot) 
heraau(decrease) 
hikaru(flash) 
akamaru(turn red) 
hieru(grow cold) 
tsuyomaru(inten- 
sify) 
utau(sing) 
arawareru(appear) 
tomeru(stop) 
hayameru(hasten) 
tsuzuku(continue) 
aobieru(tower) 
319 
54 
183 
128 
50 
22 
61 
35 
30 
29 
34 
53 
52 
54 
21 
28 
24 
29 2.1 State 
3.0 iAbstract motozuku(base) 98 
3.1 iOthers taberu(eat) 129 
Total IL433 
There are 1,209 different concepts in the 
classified concepts. 
This method was applied to the set of con- 
cepts described in Sect. 3.1 and the result is 
tabulated in Table 2. Here distribution was ob- 
tained by the classification of Chapter 5. In 
Table 2, the first digit 0, 1 and 2 in the clas- 
sification numbers roughly represent movement, 
change, and state, respectively. 
129 
4 Non-Sim~le M~tter Concepts 
Generally, non-simple matter concepts are so 
abstract in comparison with simple ones that it 
is hard to show a clear association of natural 
language with the real world. We emphasize the 
analysis of how they are composed of simple ones. 
4.1 Complex Concept A 
If two elementary matter concepts v i 
and vj(not necessarily simple ones) are 
connected according to one of the rules 
shown in Table 3 and the connected concept 
is expressed by a Japanese complex word of 
two verbs for v i and vj, it is called a '~ 
complex concept of A." The rules in Table 3 
were obtained from the investigation of a- 
bout 900 matter concepts which consist of 
two matter concepts and are expressed by a 
Japanese complex word. 
In rule XXI.I, vj(deru) is an upper- 
grade concept of vi(af~reru) and contains 
the contents of vi. Rule XXI.I is concerned 
with the whole and a part of the same mat- 
ter, while rule XXI.IIwith two different 
matters. The former is considered as a spe- 
cial case of the latter in which two mat- 
ters coincide with each other. 
Rule XXI and XXllare logical while rule 
XXI\[I is linguistic. As "cause" is one of 
the constituents in (A) in Sect. 3.1, XXI 
may be considered as a part of XX~II . 
The semantic contents of complex con- 
cept A consists of the v i and vj contents 
and their connecting relation. 
4.2 Complex Concept B 
Complex concept B consists of several 
elementary matter concepts and is usually 
expressed by a Japanese simple word. How- 
ever, no general rule can be found to con- 
nect elementary matter concepts, so a hier- 
archical analysis was made for a small num- 
ber of complex concepts of B as shown in 
Fig. 2 and Table 4. 
watasu* 
(pass) 
yuzuru ~v x (hand over 
ataer x'x 
(give)o/ u uru okuru kasu azukeru 
(sell) (present) (lend) (deposit) 
orosu * Simple matter 
(sell by wholesale) concept 
Fig. 2 A hierarchy of complex concepts of B 
From the diachronic point of view, there 
seems to be a reason why a complex concept of B 
is expressed by a simple word. The relation a- 
mong elementary matter concepts can not well be 
expressed by enumerating each verb as in the 
case of complex concept A. When one is going to 
designate matter in the real world without the 
verb identifying it, one must utter several sen- 
Table 3 
No. 
XXI 
XXI.I 
XXDK 
XXK 
XX\]I\[ 
XX \]\]I.l 
XX\]I\['\]I 
XX\]II.IK 
Connecting rules of complex concept A 
Connecting rule 
cause and 
effect 
Implication 
Cause and effect 
Logical product 
Syntactic 
connection 
Relation 
between 
s and v 
Relation 
between 
o and v 
Relation 
between 
o w and v 
Example 
(mizu-ga) afure- Jeru. 
(Water) overflow- 
comes out. 
(dareka-ga watashi -o) oshiltaosu. 
(Some one) push- 
throws (me) down. 
(sinja-ga) fushi- ogamu. 
(Believers) kneel 
down-pray. 
(akago-ga) naki- yamu. 
(A baby) cry-stops. 
(anauns~-ga genko- o) yomi-ayamaru. 
(An anouncer) read- 
misses (his manus- 
cript). (kanshu-~a sh~jin- 
o) tatakz-okosu. 
(A ~uard) knock- 
awakes (prisoners). 
Remark 
If water 
overflows, 
water comes 
out. 
If someone 
~ushes me, 
am thrown 
down. 
Believers 
kneel down 
and pray. 
That a baby 
cries stops 
An anouncer 
misses to 
read his 
manuscript. 
A guard 
awakes   
risoners y knockin~ 
tNem. 
Table 4 
Complex concept 
yuzuru(hand over) 
ataeru(give) 
~u(sell) 
orosu(sell by 
wholesale) 
oku~(present) 
Vx 
kasu (lend) 
azuke:r~. (deposite) 
An analysis of complex concepts of B 
Relation among (~)'(~)'(~) ' It elementary concepts Temporal. shift 
) ® 
,, .\[i\] 
" '*'®'® 1. ® ® 
,, ri _Dt t I % , . .L ijj  ® ® 
) @u@u@u... ) 
II '(~ 
II "@ 
O (Someone (=Pi)) has (something (=A)) )~ (P~ passes 
(A to someone (f=ie2)),~ (P2) has (A), 4~ (Pi) cele- 
brates (P2),~ (Pi) thanks (P2),~ (Pi) respects 
(e2))~ 3 (P2) has money,~ (P2) passes (money to Pi), 
(Pi) has (money), ~ (P2) returns (A to el), ~ 
(e2) uses (A), and ~ (e2) keeps (A). 
Ill Pi is higher than P2 in grade, and \[ill Pi = 
wholesaler and P2=salesman. 
6g 7) ~ )i e~ • , U and --> :logical product, logical sum and 
implication. 
vx:There is no word to represent it. 
--130-- 
Table 5 Surface contents of complex concept B 
No. Contents Example 
i0 
i0.0 
i0 .i 
i0.2 
i0.3 
11 
ii "0 
ii "I 
12 
12 "0 
12 .i 
13 
13 "0 
13 .i 
14 
14 "0 
14 .i 
14.2 
14 "3 
15 
16 
16.0 
16 .i 
i7 
17.0 
17 .i 
17 -2 
iS 
18.0 
18 .i 
59 
19.0 
19 .i 
2O 
20.0 
20 .i 
Spiritual act 
Thought.recognition 
Guess.judgement 
Respect.contempt 
Haughty.flattery 
Academic and artistic 
act 
Education.learning 
Creation 
Religious act 
Belief 
Celebration.marriage. 
funeral 
Verbal act 
Praise.blame 
Instigation.banter 
Social act 
Life 
Fostering 
Antisocial. immoral 
Promise.negotiation 
Conduct.behavior 
Labour.production 
Labour.work 
Agriculture.industry commerce 
Possesion 
Owning.abandonement 
Getting and giving. losing 
Selling and buying. 
lending and borrowing 
Investigation.meas- urement 
Investigation 
Measurement 
Domination.personal- affairs 
Domination-obedience 
Personal affairs 
Attack and defense victory and defeat 
Attack and defense 
Victory and defeat- superiority and infe- 
riority 
mitomeru(recog- 
nize) 
sassuru(guess) 
uyamau(respect) 
hikerakasu(sport~ 
oshieru(teach) 
arawasu(write a 
book) 
m$deru (visit a 
temple or shirine 
totsugu (marry) 
homeru(praise) 
iodateru(insti- igate) 
i kurasu (live) 
yashinau (bring up) 
nusumu (steal) 
i suppokasu (breake an appointment) 
aumasu(assume a prim air) 
tsutomeru(serve) 
akinau(deal in) 
y~suru(own) 
ataeru(give) 
kau (buy) 
shiraberu(inves- tigate) 
hakaru(measure) 
suberu(dominate) 
yatou(employ) 
semeru(attack) 
makasu(defeat) 
Dis 
35 
25 
18 
20 
33 
ii 
16 
16 
12 
12 
26 
26 
43 
35 
25 
35 
49 
ii 
55 
19 
24 
19 
32 
14 
26 
19 
21 Refuge.escape nigeru(escape) 22 
22 Rise and fall.pros- perity and decline 
22.0 Rise and fall horobosu(ruin) Ii 
22.1 Prosperity and de- sakaeru(prosper) 19 cline 
23 Others moyoosu(hold a 333 
meeting) 
Total I~041 
Table 6 Morpheme representing derivative operators 
L 
LI ! 
LI.I 
LI'III 
LI\[ 
No. Morpheme Example Remark 
Affix kanashi-'garu '' be sad 
Formative to 
conform affix 
Prefixal 
Suffixal 
Others 
(sad- "garu ") 
"tor~"chirakasu 
("take"-scatter 
about) 
akire-"kaeru" 
(he amazed-"re- 
turn") 
scatter about 
awfully 
be thoroughly 
amazed 
No. 
50 
50.0 
50.1 
50.2 
Table 7 
Derivative 
information 
Derivative information 
Example 
Emphasis 
Emphasis 
Do completely 
Do violently 
Respect.politeness. 
humbleness 
52 Vulgarity 
53 
53'0 
53"1 
53"2 
53"3! 
54 
54"01 
54.1 
55 
55"0 
56 
55.1 
56.0 
56.1 
56.2 
57 
58 
Poor practice.fai~ ure 
Be ill able to do 
Lose a chance to do 
Fail to do in part 
Fail to do 
Repetition.habit 
Do again 
Be used to do 
Start 
Begin to do 
Be just goingto do 
Completion 
Have finished 
Do from the begin- ning to the ena 
Have completed 
Limit 
Do until the limit 
Do throughly 
Others 
57"0 
57"1 
"tori"-chirakasu 
~'takdLscatter about) 
odoroki-"iru" 
(be surprisedJ~nte~ 
shikari-"tobasu" 
(scold-"fly") 
ossharu 
(say) 
zurakaru 
(run away) 
seme-~gumu '' 
(attack-'~gumu") 
kui-"hagureru" 
(eat-"miss") 
kaki-'@orasu" 
(write-"leak") 
fumi-"hazusu" 
(step-"take off") 
toi-"kaesu" 
(ask-"return") 
tabe-"tsukeru" 
(eat-"stick on") 
~uri- "dasu" 
(rain-"come out") 
ii- "kakeru" 
(say-"hang up") 
suri- "agaru" 
(print-"go up") 
~omi~ "t~su " 
(read-"pass through' 
~ashi- "togeru" 
i (do-"aceomplish") 
nobori-"tsumeru" i 
(climb up-"cream") 
uri-"kiru" 
(sell-"cut") 
omoshiro- "garu" 
Pt t (interesting- gar~ 
Total 
Dis 
i01 
55 
61 
5i 
31 
2 
7 
7 
16 
12 
29 
23 
5 
3 
20 
15 
209 
131 
tences. If such necessities often arise and the 
relationship is conceptualized, it will be effi- 
cient to give it a name. 
As for semantic contents, elementary matter 
concepts and their relationship form a surface 
contents. Approximately 1,000 complex concepts 
of B were investigated according to the feature 
extraction method in Sect. 3.2 and the result is 
tabulated in Table 5. 
4.3 Derivative Concept 
Some concepts possess a function of deriving 
a new concept by operating others. Matter con- 
cepts derived from operative concepts with both 
morphemic structures and derivative information 
as shown in Table 6 and 7 respectively are call- 
ed "derivative concepts." Table 7 was obtained 
from the investigation of about 700 matter con- 
cepts, most of which are expressed by a complex 
word and one concept is operative to the other. 
The derivative information is very similar to 
the modal information of auxiliary verbs, but it 
differs in that some matter concepts are operat- 
ed upon and those operations are fixed. 
5 Classification 
In order to determine whether analyses in 
Chapter 3 and 4 are good or not, we classified 
about 4,700 basic matter concepts in daily Japa- 
nese, which are listed in "Word List by Semantic 
IPreprocessing 1 
IC!assification of derivative concepts\] 
\[Classificati0nof complex concepts of A\] 
lCl@ssification of complex concepts of B\] 
\]Classif%cation of simiiar concepts I 
IClassification of ~tandard.c0ncepts I 
Fig. 3 Procedure of classification 
"V~~--~¥ U 
V T : a set of concepts of under consideration 
Vp : a class of concepts excluded by preproc- 
essing 
V V : a set of mutually different matter concepts 
V~ : a set of non-simple matter concepts 
¥C : a set of complex concepts 
V A : a class of complex concepts of A 
~B : a class of complex concepts of B 
VD : a class of derivative concepts 
V S : a set of simple matter concepts 
Vs : a class of similar matter concepts 
V b : a class of standard concepts 
Fig. 4 Relation among sets and classes 
Principles" edited by National Language Research 
Institue in Japan. 4 
5.1 Algorithm of Classificatioh 
An algorithm is introduced into the classi- 
fication, reffering Fig. 3 and 4. The elements 
or members of Vx(x=T,U,...) are denoted by Vxi(i 
=1,2,.'.) and the sum and difference in the set 
theory are denoted by + and -, respectively. 
i) Preprocessing 
For each VTi of VT, 
i.i) examine whether VTi functions with others 
or by itself. If it functions with others, then 
it is excluded from V T. 
Example. -ga~u; 
1.2) examine whether there is VTh(h<i ) which 
has the same contents as VTi and is expressed 
by the same verb as V~i. If there is such VTh, 
VTi is excluded from VT. 
Let,s denote a class of concepts excluded by 
I.I) and 1.2) by Vp and let VU=VT-Vp. 
2) Classification of derivative concepts 
For each VUi of YU, 
2.1) if VUi is expressed by a derivative word, 
it is classified as a member of term L in Table 
6. It is further classified in more detail ac- 
cording to Table 7; 
2.2) if VUi is expressed by a complex word of 
two verbs and one of these verbs is affixal, 
then it is regarded as a member of term LI in 
Table 6, and classified in more detail accord- 
ing to Table 7; 
2.3) if VUi is expressed by neither a deriva- 
tive word nor a complex word, but it is regard- 
ed as a member of one of the terms in Table 7, 
it is classified into that term. At the same 
time, it is classified into term L~ in Table 6. 
Let this class of concepts thus obtained be 
VD. 
3) Classification of complex concepts of A 
For each VUi(@VDj) of VU, if VUi is express- 
ed by a complex word of two verbs and each con- 
cept functions by itself, it is considered as a 
complex concept of A and classified according to 
Table 3. 
The class thus obtained is denoted by V A. 
4) Classification of complex concepts of B 
For each VUi(~VDj,VAk) of VU, if its con- 
tents does not belong to any term in Table 2, it 
is regarded as a complex concept of B. The class 
thus obtained is denoted by V B and subject to 
the following process: 
For each VBi , 
4.1) examine its surface structure and classify 
it according to Table i; 
4.2) examine its surface contents and classify 
it according to Table 5. 
Let V~=VD+VA+VB andVs=Vu-V ~. 
5) Classification of similar concepts 
In class V S of simple matter concepts, if 
there is a group with similar contents, choose a 
concept as the standard, then classify the re- 
132 - 
mainder as similar concepts. 
Example. Korogeru(roll)~ korobu(roll), ma- 
robu(roll), etc. are similar concepts for stand- 
ard concept korogaru(roll). 
Counter-example. Saezuru(chirp), hoeru(bark 
), unaru(roar), inanaku(neigh), etc. are not 
similar concepts for standard concept naku(cry). 
Here, it is assumed that if a certain con- 
cept is a standard concept, it is not a similar 
concept for another standard one at the same 
time. 
The class of similar concepts thus obtained 
is denoted as V s and let Vb=VS-V s. 
6) Classification of standard concepts 
For each Vbi of Vb, 
6.1) examine its structural pattern and classi- 
fy it according to Table i; 
6.2) examine its semantic contents and classify 
it according to Table 2. 
In the above process 2) through 6), one con- 
cept can be classified into two or more terms if 
necessary. 
5.2 Results and Discussion 
First, let's discuss the relation among the 
obtained classes along the abstraction process. 
There are two kinds of abstraction processes:(i) 
extracting common features from concepts as fol- 
lows; bulldog--+dog-~animal--~living thing--~thing, 
(ii) connecting several concepts to form a new 
concept as shown in complex concept B. From the 
latter viewpoint, the relation among classes is 
schematized as indicated in Fig. 5. 
Fig. 5 Relation among obtained classes 
Simple matter concepts (V b) are regarded as 
the base of matter concepts in the sense that V b 
covers the concepts of real world matter at a 
minimum and every other matter concept is led 
from V b by a rule. Two simple matter concepts 
are connected by a rule and form a little bit 
abstract concept or complex concept of A. Sever- 
al matter concepts are organized by a fairly 
complicated rule into a new abstract concept or 
complex concept of B. One of the elementary con- 
cepts in a complex concept of A changes its 
meaning diachronically and becomes a derivative 
operator. So, the system of Japanese verb con- 
cepts has its own nature--although it is a fact 
that a large part of the system is universal-- 
and is not manipulated at one level. 
Next, Table 8 indicates the distribution of 
all matter concepts. The minute distribution in 
Table 8 Distribution of matter concepts 
Class Distribution 
VS ( Vb Vs 
Vg Vc VB 
VD 
1,209 
529 
901 
951 
665 
VU 7 ¥S + ~ 4,255 Vp 
485 
V T = V U + Vp 4,740 
each class has been shown in Table 2, 5 and 7, 
respectively. Table 8 is instructive in investi- 
gating the human competence in organizing the 
language system. For example, if class Yb is re- 
garded as "primitive" concepts, number 1,209 of 
Vb does not side with Schank's classification, 9 
but with Minsky's idea. 7 From Table 2, 5 and 7, 
we can measure the degree of human concern about 
real world matter. For example, term 0.0 in Ta- 
ble 2 shows human beings are most interested in 
displacements of objects. 
Finally, we consider that every matter con- 
cept under consideration was classified satis- 
factorily supporting our analyses. 
6 Translation of Picture Pattern 
Sequences into Natural Language 
As an application of this taxonomy, system 
SUPP(Syvstem for --Understanding Picture Patterns) 
was constructed. 10-12 The overall system is 
shown in Fig. 6. 
Picture pattern sequences |Models of o < 
Hprimitive ~ i° 
~ ~- ...... \] ipietures :-- 
\[ ~"~"~'~-'=2'~7~:'='T 'I~ Syntactic 
~--,.-----, I I JMatter 
s ntactic anal met A :~,ll.!p_E!_e2_!£_ j~ 
~--- ~ ! INetwork - 
-----~------~- L for 
~.!:~ereneesl i ~ 
~_ Japanese ._ ~i 
., .......... ~ ......... , IEnglish 
Descriptive sentences -_J 
control flow ~process flow 
---~ data flow --+back-up flow 
Fig. 6 Organization of the system 
6.1 Knowledge System 
The knowledge system consists of four compo- 
nents, visual, conceptual, linguistic and the- 
saurus. The visual component contains models of 
primitive pictures and syntactic rules, which 
correspond to level 2 data in Fig. i. The rules 
are applied to picture pattern pairs called "be- 
fore-after" frame pairs. 7 
The conceptual component contains conceptual 
features, concepts, and networks of concepts, 
133 
which correspond to level 3 through 5 data, res- 
pectively. A matter concept is expressed by 
\[v : ClC2.'.cldl(El)d2(E2)'..dm(Em)\] (B) 
where each c i denotes a feature of matter itself 
and is associated with a syntactic rule mention- 
ed above. Each dj( ) denotes the case or roll of 
a constitnent and must be filled by a specific 
instance or concept of constituent. Features Ej( 
=ejlej2...ejn ) specify the conditions its as- 
slgnment must meet. A network is constructed a- 
mong similar matter concepts. 
The linguistic component consists of dic- 
tionaries for the production of Japanese and 
English sentences. The thesaurus component con- 
tains all the classified concepts in Chapter 5 
and supports the development of other components. 
6.2 Translation Process 
A sequence of picture patterns, or two-di- 
mensional line drawings(handwriting is allowed), 
is input at time t0,tl,...,tn. A picture pattern 
at ti(0~i~n-l) is paired with the one at ti+l, 
and processed as follows: 
i) Primitive picture recognition and syntactic 
analysis 
The picture pattern reader is a curve fol- 
lower that traces line segments by octagonal 
scanning. The recognizer is based on Evans's 
matching program for graph-like line drawings 
but is improved to handle noisy ones. 13 
The syntactic analyzer A decomposes the 
complex picture, in which two or more primitive 
pictures may intersect or touch each other, and 
recognizes them according to Gestalt criteria. 
The syntactic analyzer B performs Boolean opera- 
tions on quantized primitive pictures to check 
such a relation as "MAN INSIDE HOUSE." The syn- 
tactic analyzer C performs numerical operations 
on the data such as coordinates and transforma- 
tional coefficients of primitive pictures. 
2) Semantic analysis and inference 
The semantic analyzer detects the meaning of 
matter-centred change in picture pattern pairs 
by top-down analysis. Suppose that matter \[v(s, 
oa): ClC2...Clds(es)doa(eo~)\] is directed by the 
Inference. The analyzer assigns the role of s to 
one of the primitive pictures, say Ps, after 
checking whether Ps meets e s. It assigns the 
role of o~ to another primitive picture Poa in 
the same way. Then it analyzes each c i by call- 
ing a correspondent sub-program in the syntactic 
analyzer B or C. If all the analyses end in suc- 
cess, the meaning of v(s,oa) is detected, The 
present Inference makes inferences about all the 
similar concepts in the network in depth-first 
order, directing each matter concept at a node 
to the semantic analyzer. 
Finally, the synthesizer produces Japanese 
and English simple sentences. 
6.3 Experiments 
All the programs except the picture pattern 
reader are written in Fortran and run under the 
OS~/VS of the FACOM 230-38S medium scale com- 
puter at Oita University. Running with the syn- 
tactic analyzer B and C, the semantic analyzer 
occupies approximately 200K bytes of core. Mem- 
ory usage for all the dictionaries except the 
thesaurus component amounts to approximately 90K 
bytes. 
Fig. 7 and 8 indicate an example of the 
recognition of a primitive picture and the 
translation of a picture pattern pair, respec- 
tively. It took 47 seconds to recognize "bird\[l\]" 
in Fig. 7 and 60 seconds to analyze and infer 
the meanings of matter after the recognition of 
primitive pictures in Fig. 8. 
Katz and Fodor pointed out the three prob- 
lems of a semantic theory: (i) Semantic ambigui- 
ty, (ii) Semantic anomaly, and (i/) Paraphrase. 
Pl P2 P8 Pi9 ~.--~/ Pi8 
P3p~5~ 1 ql q 5ql~(~ 3 
P9( Pll ~7P16q/2 <( ~qS~q 
Pl0~ P13 
(a) Preprocessed (b) The model of 
primitive picture "bird\[l\]" 
i. Pattern matching 
\[P7 :q5\], \[P9:q14 \], \[Pl9"q~\]," \[PS:q~\], \[P6:ql~\], 
\[Pl0:ql3 \]' \[PlS:q7 \], \[P5:q2 \]'tpll:ql2 \]' \[P17: q8 J , \[P12:q9 \] 
\[Pl6:qlO\], \[Pl2:q9\],\[Plq:qll\] 2. Transformation and similarity 
Translation: (524.4, 483.7), scaling: 2.173 
times, rotation: 2.290 radian, reflection: in Y 
-axis. Similarity: 0.718(<_1) 
Fig. 7 Recognition of "bird\[l\]" 
Fig. 8 
t=t 0 t=t I 
(a) Input picture pattern pair 
i) TORI\[I\] GA UTSURU. 
THE BIRD\[l\] SHIFTS. 
2) TORI \[ 1 \] GA SUSUMU. 
THE BIRD\[l\] MOVES ON. 
3) TORI\[2\] GA TOBU. 
THE BIRD\[l\] FLIES. 
4 ) TORI\[ 1 \] GA KI NI FURERU. 
THE BIRD\[l\] TOUCHES THE TREE. 
5 ) TORI \[ 1 \] GA KI NI TSUKU. 
THE BIRD\[l\] STCKS TO THE TREE. 
6) TORI\[1\] GAfI NI NORU\[2\]. 
THE BIRD\[l\] GETS\[2\] ON THE TREE. 
7) To~r\[7\] <;A x7 ~I ~OSH¢~U. 
THE BIRD\[\].\] LEANS OVER THE TREE. 
(b) Output sentences 
Translation of a picture pattern pair 
134 
As for (i) it is important to enumerate all the 
readings of the input picture pattern pair. The 
output sentences in Fig. 8 shows SUPP under- 
stands to a fair degree the change in the input. 
As for (ii) the ability in detecting semantic 
anomaly is important. SUPP checks it by Ej in ( 
B) in Sect. 6.1, but a little bit anomalous sen- 
tence 5) or 7) is output because the constructed 
dictionary of matter concepts is slightly insuf- 
ficient. As for (JJi) the ability in paraphrasing 
sentences is needed. Output sentences 4) through 
7) are an analytical paraphrase of "THE BIRD\[i\] 
PERCHES ON THE TREE" although SUPP has no know- 
ledge about "perch." 
7 Conclusions 
A taxonomy of Japanese matter concepts has 
been described. It is summarized as follows: 
Simple matter concept 
Standard concept 
fstructural pattern ; types 12 
~semantic contents ; 20 features 
Similar concept 
Non-simple matter concept 
Complex concept A 
connecting rule 3 types 
semantic contents ; each contents of 
elements and their connecting re- 
lation 
Complex concept B 
connecting rule ; different in each 
concept 
surface contents ; 14 features 
Derivative concept ; 
derivative operator ; types 3 
derivative information ; 9 features 
This taxonomy has made clear the outline of 
the system of all matter concepts in daily Japa- 
nese, and by SUPP picture pattern understanding 
research has come closer to natural language un- 
derstanding research. 
Acknowledsement 
The author started his investigation of Jap- 
anese matter concepts and the development of 
SUPP some ten years ago when he was at Kyushu 
University. The author wishes to express grati- 
tude to Prof. T.Tamati of Kyushu University for 
his kind guidance and material support. 

References 

\[i\] 0kada, N. and Tamati, T.: Semantic Informa- 
tion of Natural Language and its Extraction and 
Classification, Trans. IECE, Japan, 52-C, i0, p. 
363(0ct. 1969). 

\[2\] Okada, N. and Tamati, T.: An Analysis and 
Classification of "Simple Matter Concepts" for 
Natural Language and Picture Interpretation, 
Trans. IECE, Japan, 56-D,9,p.523-530(Sep~ 1973). 

\[3\] Okada, N. and Tamati, T. : An Analysis and 
Classification of "Non-Simple Matter Concepts" 
for Natural Language and Picture Interpretation, 
Trans. IECE, Japan, 56-D,10,p.591-598(0cL 1973). 

\[4\] National Language Research Institute(ed.): 
"Word List by Semantic Principles," Syuei Syup- 
pan, Tokyo, 1964. 

\[5\] Roget, P.(Browning, D.C.(ed.)):"Thesaurus of 
English Words and Phrases," J.M. Dent and Sons 
Ltd, London, 1971. 

\[6\] Chomsky, N.: "Aspects of the Theory of Syn- 
tax," The M.I.T. Press, Cambridge, Mass., 1965. 

\[7\] Minsky, M,: A Framework for Representing 
Knowledge, in Winston, P.H.(ed.),"The Psychology 
of Computer Vision," McGraw-HillBook Co., New 
York, 1975. 

\[8\] Schank,R.C. and Abelson, R.P.: Scripts, 
Plans, and Knowledge, Proc. 4IJCAI, Tiblisi, 
Georgia, 1975. 

\[9\] Schank, R.C. and Rieger, C.T.: MARGIE: Memory, 
Analysis, Response Generation, and Inference on 
English, ~oc. 3IJCAI, Stanford, 1973. 

\[i0\] Okada, N. and Tamati, T.: Interpretation of 
the Meaning of Picture Patterns and its Descrip- 
tion in Natural Language--Primitive Picture Re- 
cognition and Syntactic Analysis, Trans. IECE, 
Japan, J59-D, 5, p.323-330(May 1976). 

\[ii) Okada, N. and Tamati, T.: Interpretation of 
the Meaning of Moving Picture Patterns and its 
Description in Natural Language---Semantic Analy- 
sis, Trans. IECE, Japan, J59-D , 5, p.331-338(May 
1976). 

\[12\] Okada, N.: SUPP: Understanding Moving Pic- 
ture Patterns Based on Linguistic Knowledge, 
Proc. 6IJCAI, Tokyo, 1979, 

\[13\] Evans, T.G.: A Program for the Solution of 
Geometric Analogy Intelligence-Test Questions, 
in Minsky, M.(ed.): "Semantic Information Pro- 
cessing,"The M.I.T. Press, Cambridg~Mass., 1968. 

\[14\] Katz, J.J. and Fodor, J.A.: The Structure 
of a Semantic Theory, Language, 39, 2, p.170-210 (1963). 
