Semantic Network Array Processor as a 
Massively Parallel Computing Platform for 
High Performance and Large-Scale Natural Language Processing* 
Hiroaki Kitano 
Center for Machine Translation 
Carnegie Mellon University 
Pittsburgh, PA 15213 U.S.A. 
Abstract 
This paper demonstrates the utility of 
the Semantic Network Array Processor 
(SNAP) as a massively parallel platform 
for high performance and large-scale nat- 
ural language processing systems. SNAP 
is an experimental massively parallel ma- 
chine which is dedicated to, but not lim- 
ited to, the natural language processing us- 
hag semantic networks. In designing the 
SNAP, we have investigated various natu- 
ral language processing systems and the- 
ories to determine the scope of the hard- 
ware support and a set of micro-coded 
instructions to be provided. As a re- 
suit, SNAP employs an extended marker- 
passing model and a dynamically modi- 
fiable network model. A set of primi- 
tive instructions is micro-coded to directly 
support a parallel marker-passing, bit- 
operations, numeric operations, network 
modifications, and other essential func- 
tions for natural language processing. This 
paper demonstrates the utility of SNAP 
for various paradigms of natural language 
processing. We have discovered that the 
SNAP provides milliseconds or microsec- 
onds performance on several important ap- 
plicatious such as the memory-based pars- 
ing and translation, classificatlon-based 
parsing, and VLKB search. Also, we ar- 
gue that there are numerous opportunities 
in the NLP community to take advantages 
of the comlmtational power of the SNAP. 
1. Introduction 
In order to accomplish the high-performance natural 
language processing, we have designed a highly par- 
allel machine called Semantic Network Array Proces- 
sor (SNAP) \[Lee and Moldovan, 1990\]. The goal of 
our project is to develop and test the validity of the 
massively parallel machine for high performance and 
larg-scale natural language processing. Thus, the ar- 
chitecture of the SNAP was determined reflecting ex- 
tensive analysis of basic operations essential to the 
"This research is Bupported by the National Science 
Foundation under grant MIP-9009111 and MIP-9009109, 
and conducted as a part of IMPACT (InternationM 
Consortium for Massively Parallel Advanced Computing 
Technologies) 
Dan Moldovan 
Department of Electrical Engineering Systems 
University of Southern California 
Los Angeles, CA 90089-1115 U.S.A. 
natural language processing. As a result of the in- 
vestigation, we have decided to employ an extended 
marker-passing model and a dynamically modifiable 
network. Also, a set of primitive instructions is micro- 
coded to directly support essential operations in nat- 
ural language systems. 
Several approach can be taken to use SNAP as a 
platform for natural language processing systems. We 
can fully implement NLP system on SNAP, or we can 
speed up existing systems by implementing compu- 
tationally expensive part on SNAP. We have hnple- 
mented some of these approaches on SNAP, mid ob- 
tained extremely high performance (order of millisec- 
onds for given tasks). 
In this paper, we describe the design philosophy 
and architecture of SNAP, and present several ap- 
proaches toward high performance natural language 
processing systems on SNAP. 
2. SNAP Architecture 
2.1. Design Philosophy of SNAP 
The Semantic Network Array Processor (SNAP) is 
a highly parallel array processor fully optindzed tbr 
semantic network processing with a marker-passing 
mechanism. The fundermental design decisions arc 
(1) a semantic network as a knowledge representation 
scheme, and (2) parallel marker-passing as an infer- 
ence mechauism. 
First, the use of a semantic network as a represem 
tation scheme can be justified from the fact that most 
of the representation schemes of current AI and NLP 
theories (such as frame, feature structure, sort hier- 
archy, systemic choice network, neural network, etc.) 
can be mapped onto semantic networks. Also, tlmre 
are numbers of systems and models which directly use 
semantic networks \[Sown, 1991\]. 
Second, the use of marker-passing can be jus- 
tified from several aspects. Obviously, there are 
many AI and NLP models which use some form 
of marker-passing as the central computing princi- 
ple. For example, there are significant number of re- 
search being done on word-sense disambiguation as 
scene in Waltz and Pollack 1985\] Itendler, 1988\], 
\[Hirst, 1986, \[Charniak, 1983\], \[Tomabechi, 1987, 
etc. All of them assume passing of markers or val- 
ues among nodes interconnected via some types of 
links. There are studies to handle syntactic con- 
ACRES DE COLING-92. NANaXS, 23-28 AO~" 1992 8 1 3 Paoc. ov COLING-92. NAI, rVES, AUG. 23-28, 1992 
tsmlttamtm4 ~ettmmant 
|NAP.1 ~ Cam~ oan@~a¢ c.J ~t~ ss^p 
Figure I: SNAP-1 Architecture 
straints using some type of networks which can be 
mapped onto semantic networks. Recent studies on 
the Classification-Based Parsing \[Kasper, 1989\] and 
the Systemic Choice Network \[Carpenter and Pollard~ 
1991\] assume hierarchical networks to represent var- 
ions linguistic constraints, and the search on these 
networks can be done by marker-passing. Also, there 
are more radical approaches to implement entire natu- 
ral language systems using parallel marker-passing as 
seen in \[Norvig, 1986\], \[Riesbeck and Martin, 1985\], 
\[Tomabechi, 1987\], and \[Kitano, 1991\]. There are, 
however, differences in types of information carried 
in each marker-passing model. We will describe our 
design decisions later. 
As reported in \[Evett, at. al., 1990\], however, serial 
machines are not suitable for such processing because 
it causes performance degradation as a size of seman- 
tic network increases. There are clear needs for highly 
parallel machines. The rest of this section provides a 
brief overview of the SNAP architecture. 
2.2. The Architecture 
SNAP consists of a processor array and an array con- 
troller (Figure 1). The processor array has processing 
cells which contain the nodes and hnks of a semantic 
network. The SNAP array consists of 160 process- 
ing elements each of which consists of a TMS320C30 
DSP chip, local SRAM, etc. Each processing ele- 
ments stores 1024 nodes which act as virtual proces- 
sors. They are interconnected via a modified hyper- 
cube network. The SNAP controller interfaces the 
SNAP array with a SUN 3/280 host and broadcasts 
instructions to control the operation of the array. The 
instructions for the array are distributed through a 
global bus by the controller. Propagation of markers 
and the execution of other instructions can be pro- 
ceased simultaneously. 
2.3. Parallel Marker-Passing 
In the SNAP, content of the marker are: (1) bit- 
vector, (2) address, and (3) numeric value (integer 
or floating point). In SNAP, the size of the marker 
is fixed. According to the classification in \[Blelloch, 
1986\], our model is a kind of Finite Message Pass- 
ing. There are types of marker-, or message-, pa~ing 
that propagates feature structures (or graphs)~ which 
are called Unbounded Message Passing. Although we 
have extended our marker-passing model from the tra- 
ditional bit marker-passing to the complex marker- 
passing which carries bits, address, and numeric val- 
ues, we decided not to carry unbounded messages. 
This is because propagation of feature structures and 
heavy symbolic operations at each PE are not prac- 
tical assumptions to make, at least, on current mas- 
sively parallel machines due to processor power, mem- 
ory capacity on each PE, and the communication bot- 
tleneck. Propagation of feature structures would im- 
pose serious hardware design problems since the size 
of the message is unbounded, which means that the 
designer can not be sure if the local memory size is 
sufficient or not until the machine actually runs some 
applications. Also, PEa capable of performing oper- 
ations to manipulate these messages (such as unifi- 
cation) would be large in physical size which causes 
assembly problems when thousands of processors are 
to be assembled into one machine. Since we decide 
not to support unbounded message passing, we decide 
to support functionalities attained by the unbounded 
message passing by other means such as sophisticated 
marker control rules, dynamic network modifications, 
etc. 
2.4. Instruction Sets 
A set of 30 high-level instructions specific to semantic 
network processing are implemented directly in hard- 
ware. These include associative search, marker set- 
ting and propagation, logical/arithmetic operations 
involving markers, create and delete nodes and re- 
lations, and collect a list of nodes with a certain 
marker set. Currently, the instruction set can be 
called from C language so that users can develop ap- 
plications with an extended version of C language. 
From the programming level, SNAP provides data- 
parallel programming environment similar to C* of 
the Connection Machine \]Thinking Machines Corp., 
1989\], but specialized for semantic network process- 
ing with marker passing. 
Particularly important is the marker propagation 
rules. Several marker propagation rules are provided 
to govern the movement of markers. Marker propa- 
gation rules enables us to implement guided, or con- 
straint, marker passing as well as unguided marker 
passing. This is done by specifying the type of links 
that markers can propagate. The following are some 
of the propagation rules of SNAP: 
e Seq(rl, r~): The Seq (sequence) propagation rule 
allows the marker to propagate through rl once 
then to r~. 
• Spread(rl,r2) : The Spread propagation rule al- 
lows the marker to travel through a chain of rl 
links and then r~ links. 
* Comb(rl,r~) : The Comb (combine) propagation 
rule allows the marker to propagate to all rl and 
r~ links without limitation. 
2.5. Knowledge Representation on SNAP 
SNAP provides four knowledge representation ele- 
ments: node, link, node color and link value. These 
elements offer a wide range of knowledge representa- 
tion schemes to be mapped on SNAP. On SNAP) a 
concept is represented by a node. A relation can be 
represented by either a node called relation node or 
AcrEs DE COLING-92, NANTES. 23-28 AOt)r 1992 8 1 4 PROC. OF COLING-92, NANTES. AUG. 23-28. 1992 
a link between two nodes. The node color indicates 
the type of node. For example, when representing USC 
is in Los Angeles and CW0 ie in Pittsbnrgh~ we 
may assign a relation node for IN. The IN node is 
shared by the two facts. In order to prevent the wrong 
interpretations such as USC in Pittsburgh and CSll 
in Lea Angeles, we assigu IN#I and IN#2 to two 
distinct IN relations, and group the two relation nodes 
by a node color IN. Each lhlk has assigned to it a link 
value which indicates the strength of interconcepts re- 
lations. This link value supports probabilistic reason- 
ing and connectionist-like processing. These four basic 
elements allow SNAP to support virtually any kind 
of graph-based knowledge representation formalisms 
such as KL-ONE \[Braehman and Schmolze, 1985\], 
Conceptual Graphs \[Sown, 1984\], KODIAK \[Wilen- 
sky, 1987\], etc. 
3. The Memory-Based Natural 
Language Processing 
Memory-baaed NLP is an idea of viewing NLP as a 
memory activity. For example, parsing is considered 
as a memoryosearch process which identifies similar 
eases in the past from the memory, and to provide 
interpretation based on the identified case. It can be 
considered as an application of Memory-Baaed l~.ea- 
soning (MBR) \[Stm~fill and Waltz, 1986\] and Case- 
Based Reasoning (CBR) \[Riesbeck and Schank, 1989\] 
to NLP. This view~ however, counters to traditional 
idea to view NLP as arl extensive rule application pro- 
cess to build up meaning representation. Some mod- 
els has been proposed in this direction, such as Direct 
Memory Access Parsing (DMAP) \[Riesbeck and Mar- 
tin, 1985\] and q~DMDIALOO \[Kitano, 1991\]. For ar- 
guments concerning superiority of the metnory-based 
approach over the traditional approach, ace \[Nagao, 
1984\], \[Riesbeck and Martin, 1985\], and \[Sumita and 
\]\[ida, 1991\]. 
DMSNAP is a SNAP implementation of the 
(I)DMDIALOG speech-to-speech dialogue translation 
system which is based on, in part, the memory-based 
approach. Naturally, it inherits basic ideas and mech- 
anisms of the ~DMDIALOG system such as a memory- 
based approach to natural language processing and 
parallel marker-passing. Syntactic constraint network 
is introduced in DMSNAP whereas ODMDIALOG has 
been assuming unification operation to handle linguis- 
tic processing. 
DMSNAP consists of the nlemory network, syntac- 
tic constraint network, and markers to carry out in- 
ference. The memory network and the syntactic con- 
straint network are compiled from a set of grammar 
rules written for DMSNAP. 
Memory Network on SNAP The major types of 
knowledge required for language translation in DM- 
SNAP are: a lexicon, a concept type hierarchy, con- 
cept sequences, and syntactic constraints. Among 
them, the syntactic constraints are represented in 
the syntactic constraint network, and the rest of the 
knowledge is represented in the memory network. The 
memory network consists of various types of nodes 
such as concept sequence class (CSC), lexical item 
node* (LEX), concept nodes (CC) and others. Nodes 
are connected by a number of different links such as 
concept ahstraction links (ISA), expression links for 
both source language and target language (ENG and 
JPN), Role links (ROLE), constraint links (CON- 
STRAINT), contextual llnk~ (CONTEXT) and oth- 
ers. A part of the menmry network is shown in Figure 
2. 
Markers The processing of natural language on a 
marker-propagation architecture requires the creation 
and movement of markers on the memory network. 
The following types of markers are used: (1) A- 
Markers indicate activation of nodes. They propa- 
gate ttlrough ISA links upward, carry a pointer to 
tile source of activation axtd a cost measure, (2) P- 
Markers indicate the next possible nodes to be acti- 
vated. They are initially placed on the first element 
nodes of the CSGs, and move through NEXT link 
where they collide with A-MARKERs at tile element 
nodes, (3) G-Markers indicate activation of nodes in 
tile target language, They carry pointers to the lexi- 
eal node to he lexicalized, and propagate througtl ISA 
links upward, (4) V-Markers indicate current state of 
the verbalization. When a V-MARKER collides with 
the G-MARKER, the surface string (which is specified 
by the pointer in the G-MARKER) is verbalized, (5) 
G-Markers indicate contextual priming. Nodes with 
C-MAItKERs are contextually primed. A C-MARKER 
moves from the designated contextual root node to 
other contextually relevant nodes through contextual 
links, and (6) SC-Markers indicate active syntax con- 
attaints, and primed and/or inhibited nodes by cur- 
rently active syntactic constraints. It also carries 
pointer to specific nodes. There are some other mark- 
era used for control process and tinting; they are not 
described here. 
The parsing algorithm is sinular to the shift-reduce 
parser except that our algorithms handle ambiguities, 
parallel processing of each hypothesis, and top-down 
predictions of possible next input symbol. The gen- 
eration algorithm implemented on SNAP is a version 
of the lexically guided bottom-up algorithm which is 
described in Kitano 1990\]. Details of the algorithm 
is described in Kitano et. al., 1991b. 
DmSNAP can handle various linguistic l)henomena 
such as: lexical ambiguity, structural ambiguity, ref- 
erencing (pronoun referenee~ definite noun reference, 
etc), control, and unbounded dependencies. Linguis- 
tically complex phenomena are handled using the syn- 
tactic constraint network (SCN). The SCN enables the 
DmSNAP to process sentences involving unbounded 
dependencies, controls without passing feature struc- 
tures. Details of the SCN is described in \[Kitano et. 
ah, 1991h\]. One notable feature of DmSNAP is its 
capability to parse and translate sentences in context. 
In other words, DmSNAP can store results of previ- 
ous sentences and resolve various levels of ambiguities 
using the contextual information. Examples of sen- 
tences which DmSNAP can handle is shown below. 
it should he noted that each example consists of a 
set of sentences (not a single sentence isolated from 
the context) ill order to denmnatrate the contextual 
processing capability of the DMSNAP. 
ACRES DE COLING-92, NANTES, 23-28 Ao~r 1992 8 1 5 PRec. OF COLING-92, NANTES. AUO. 23-28, 1992 
; '~ ln,~,nc. Nod. 
c-e .m • - 
:~o-q: ' -co~-J 
Figure 2: Part of Memory Network 
Sentence Length Time at 
(words) 10 MHz ( ..... ) 
s2; He is at ... 4 0.65 
s3: He said that ... 10 1.50 
sS: Eric build ...' 5 0.55 
s6: Juntas found .. 6 1.00 
s8: Juntae solved ... 7 1.65 
Table 1: Execution times for DmSNAP 
Example I 
sl John wanted to attend Collng-92. 
m2 He is at the conference. 
s3 He said that the quality of the paper is superb. 
Example II 
s4 Dan planned to develop a parallel processing 
computer. 
s5 Eric built a SNAP simulator. 
s6 Juntae found bugs in the simulator. 
s7 Dan tried to persuade Eric to help Juntae modify 
the simulator. 
s8 Juntae solved a problem with the simulator. 
s9 It was the bug that Juntae mentioned. 
These sentences in examples are not all the sen- 
tences which DMSNAP can handle. Currently, DM- 
SNAP handles a substantial portion of the ATR con- 
ference registration domain (vocabulary 450 words, 
329 sentences) and sentences from other corpora. 
The following are examples of translation into 
Japanese generated by the DmSNAP for the first set 
of sentences (sl, s2 and s3): 
tl Jon ha koringu-92 ni sanka shitakatta. 
t2 Kate ha kalgi niiru. 
t3 Kare ha ronbun no shitsu ga subarnshli to itta. 
DMSNAP completes the parsing in the order of 
milliseconds. Table 1 shows parsing time for some of 
the example sentences. 
F3 
gender male 
number singular 
person 3rd 
F1 
I gende, male \[ numbv~ singular 
person 3rd 
Figure 3: A part of a simple example of classification 
lattice 
4. Classification-Based Parsing 
Classification-Based Parsing is a new parsing model 
proposed in \[Kasper, 1989\]. In the classification-based 
parsing, feature structures are indexed in the hier- 
archical network, and an unifiability of two feature 
structures are tested by searching the Most Specific 
Subsumer (MSS). The unification, a computationally 
expensive operation which is the computational bot- 
tleneck of many parsing systems, is replaced by search 
in the lattice of pre~indexed feature structures. 
For example, in Figure 3, the feature structure 
F3 is a result of successful unification of the feature 
structure F1 and F2 (F3 = F1 tA F2). All feature 
structures are pro-indexed in a lattice so that the uni- 
fication is replaced by an intersection search in the 
lattice with complex indexing. To carry out a search, 
first we set distinct markers on each feature structures 
F1 and F2. For example, set marker M1 on F1, and 
M2 on F2. Then, markers M1 and M2 propagate up- 
ward in the lattice. M1 and M2 first co-exist at F3. 
The most simple program (without disjunctions and 
conjunctions handling) for this operation follows: 
sot _marker (M1 ,~t) ; 
n~_marker (M2 ,f2) ; 
propagate(M1 ,M1 ,UP,I/P pSPREAD) ; 
propagate (I(2,M2,UP,UP,SPREID) ; 
marker_and ( M 1, M2 j M3) ; 
propagate (M3, re_trap, UP ,UP, SPREAD ) ; 
cond_clear marker (m_tmp, M3) ; 
collect nodes (M3) ; 
Of course, nodes for each feature structure may 
need to be searched from a set of features, instead 
of direct marking. In such a case, a set of markers 
will be propagated from each node representing each 
feature, and takes disjunction and conjunction at all 
nodes representing a feature structure root. This op- 
eration can be data-parallel. 
There are several motivations to use classification- 
based parsing, some of which are described in \[Knsper, 
1989\]. The efficiency consideration is one of the major 
reasons for using classification-based parsing. Since 
over 80% of parsing time has been consumed on uni- 
fication operations, replacing unification by a faster 
and functionally equivalent method would substan- 
tially benefit the overall performance of the system. 
The classification-based parsing is efficient because (1) 
it maximizes structure sharing, (2) it utilizes indexing 
dependencies, and (3) it avoids redundant computa- 
tions. However, these advantages of the classification- 
AcrEs DE COLING-92, NANTES, 23-28 hO'J'f 1992 8 1 fi PROC. OF COLING-92, NANTES, AUG. 23-28, 1992 
Time (u aec.) 
7$ 
• • 
65 2 3 4 5 o 7 8 Fan-Out 
Time (~ see.) 
32 04 128 256 Coneeptl 
Figure 4: Retrieval Performance on Classification Net- 
work 
based parsing can not be fully obtained if the model 
was implemented on the serial machine. This is be- 
cause a search on complex index lattice would be 
computationally expensive for serial machines. Ac- 
tually, the time-complexity of the sequential classi- 
fication algorithm is O(Mn2), and that of the re- 
trieval algorithm is O(R,,~JogM), where M is a num- 
ber of concepts, n is an average number of prop- 
erty links per concept, R,.c is an average number of 
roleset relations for one concept. We can, however, 
circumvent this problem by using SNAP. Theoreti- 
cally, time-complexity of the classification on SNAP is O(loggo~,M), 
and that of the parallel retrieval is O(FinDa., + 17~), 
where Fo~t is an average fan-out 
(average number of suhconcepts for one concept), J~ 
is an average fan-in (average number of superconcept 
for one concept), and D.~. is an average depth of the 
concept hierarchy \[Kiln and Moldovan, 1990\]. 
In our model, possible feature structures are pre- 
computed and indexed using our classification algo- 
rithms. While a large set of feature structures need 
to be stored and indexed, SNAP provides sufficiently 
large memory/processor space to load an entire fea- 
ture structure lattice. It is analogous to the idea be- 
hind the memory-based parsing which pre-expand all 
possible syntactic/semantic structures. Here again, 
we see the conversion of tlme-complexity into space- 
complexity. 
Figure 4 shows performance of retrieval of clas- 
sitleation lattice with varying fan-out and size. The 
clock cycle is 10 MHz. It demonstrates that we can 
attain micro-seconds response for each search. Given 
the fact that the fastest unification algorithm, even on 
the parallel machines, takes over few milliseconds per 
unification, the performance obtained in our experi- 
ment promises a significant improvement in parsing 
speed for many of the unification-baaed parsers by re~ 
placing unification by classification-based approach. 
5. VLBK Search: Integration with 
the Knowledge-Based Machine 
q_¥anslation 
Language processing is a knowledge-intensive process. 
Knowledge-Based Machine Trans- 
lation (KBMT)\[Goodman and Nirenberg, 1991\] has 
been proposed and developed baaed on the assumption 
that intensive use of linguistic and world knowledge 
would provide lfigh quality automatic trmmlation. 
One of the central knowledge sources of the KBMT 
is the ontological hierarchy which encodes abstraction 
hierarchies of concepts in the given domain, prop~ 
erty information of each concept, etc. When a parser 
creates ambiguous parses or when some parts of the 
meaning representation (as represented in an interlin- 
gun) are missing, this knowledge source is accessed to 
disambiguate or to fill-in missing information. 
However, as the size of the domain scales up, access 
time to the knowledge source grows to the extent that 
cost-effective bulk processing would lint be possible. 
For exmaple, \[Evett, el. al., 1990\] reports that ac- 
cess to large frame systems on serial computers have a 
time-complexity ofO(M x B 't) where M is the number 
of conjuncts in the query, B is the average branching 
factor in the network, and d is the deptb of the net- 
work. Thus, even a simplest form of search takes over 
6 seconds on a VLKB with 28K nodes measured on 
a single user mode VAX super nfini-computer. Since 
such search on a VLKB must be performed several 
times for each parse, the performance issue would be 
a major concern. Considering the fact that VLKB 
projects such as CYC \[Lenat and Guha, 1990 and 
EDR \[EDR, 1988\] aim at VLKBs containing over a 
million concepts, the performance of VLKB search 
would be an obvious problem in practical use of these 
VLKBs. In tile massively parallel machines such as 
SNAP, we should be able to attain time-complexity of O(D + M) 
\[Evett, et. al., 1990\]. 
We have carried out experiments to measure KB 
access time on SNAP. Figure 5 shows the search time 
for various size of VLKBs ranging from 800 to 64K 
nodes. Performance was compared with SUN-4 anti 
the CM-2 connection machine. SNAP-1 consistently 
outperformed other machines (performance curve of 
SNAP-1 is hard to see in the figure as it exhibited 
execution time far less than a second. 
6. Other Approaches 
One clear extension of the currently implemented 
modules is to integrate the classification-baaed parsing 
and the VLKB search. The classification-baaed pars- 
ing carry out high performance syntactic analysis and 
the VLKB search would impose semantic constraints. 
Integration of these two would require that the SNAP- 
1 to have a multiple controller because two different 
marker control processes need to he mixed and exe- 
cuted at the same time. Currently SNAP-1 has only 
one controller. This would be one of the major items 
for tile up-grade of the architecture. However, the 
performance gain by this approach would be signifi- 
cant and its in,pact can be far reaching because a lot 
of current NLP research has been carried out on the 
ACflLS DE COLING-92, NANTES, 23-28 AOt~rr 1992 8 1 7 I'ROC. OF COLING-92, NAN-t~S, Autl. 23-2g, 1992 
VLKII Retrieval in PACE Benchmark 
+++ alooo .............. -.-- 
i~00 . -- 
I ' 
. 
Iso00 ......... ~ -- 
12OOO ............ : -- + 
tlooo ~ ........ 7 
100 00 +I ....... _ ,,'L- ....... ,mm ..... ~ ...... 
m00 ........ : ...... 7OO0 ................ 
+0.+0 +~ ........ ," ........ :+\] 
:+7'"+ -~) +am -- + -- ~ -- -- -- :+oal ............ +m ......... NAP-1 
ooo lm 2o+ +~ 4oo +¢~ +ix, 
Figure 5: Retrieval time vs. KB size 
framework of the unification-based grammar formal- 
ism and use VLKBs as major knowledge sources. 
A more radical approacl h however rooted in the tra- 
ditional model is to fully map the typed unification 
grammars \[Emele and Zajac, 1990 on the SNAP. The 
typed unification grammar is based on the Typed Fea- 
ture Structure (TFS) \[Zajac, 1989\] and HPSG \[Pollard 
and Sag, 1987\], and represents all objects in TFS. 
Objects includes Phrasal Sign, Lexical Sign, general 
principles such as the "Head Feature Principle", the 
"Subcat Feature Principle", grammar rules such as 
the "Complement Head Constituent Order Feature 
Principle," the "Head Complements Constituent Or- 
der Feature Principle," and lexical entries. The lexi- 
cal entries can be indexed under the lexical hierarchy. 
In this apporach, all linguistic knowledge is precom- 
piled into a huge network. Parsing and generation 
will be carried out as a search on this network. We 
have not yet complete a feasibility study for this ap- 
proach on SNAP. However, as of today, we consider 
this approach is feasible and expect to attain single- 
digit millisecond order performance on an actual im- 
plementation. The dynamic network modification, ad- 
dress propagation, and marker propagation rules are 
especially useful in implementing this approach. 
Natural language processing model on semantic 
networks such as \[Norvig, 1986\], SNePS \[Neal and 
Shapiro, 1987\], and TRUMP, KING, ACE~ and 
SOISOR at GE Lab. \[Jacobs, 1991\] should fit well 
with the SNAP-1 architecture. For \[Norvig, 1986\], 
SNAP provides floating point numbers to be propa- 
gated. As for SNePS, the implementation should be 
trivial, yet we are not sure the level of parallelism gain 
by the SNePS model. When the parallelism was found 
to be low, the coarse-grain processor may fit well with 
this model. Although we do not have space to discuss 
in this paper, there are, of course, many other NLP 
and AI models which can be implemented on SNAP. 
7. Conclusion 
In this paper, we have demonstrated that semantic 
network array processor (SNAP) speeds up various 
natural language processing tasks. We have demon- 
atrated this fact using three examples: the memory- 
based parsing, VLKB processing, and Classification- 
based parsing. 
in the memory-based parsing approach, we have at- 
tained the speed of parsing in the order of milliseconds 
without making substantial compromises in linglfistic 
analysis. To the contrary, our model is superior to 
other traditional natural language processing models 
in several aspects, particularly, contextual processing. 
Next, we have applied the SNAP architecture for a 
new classification-based parsing model ltere, SNAP 
iB used to search tile MSS to test tile unifiability of 
the two feature graphs. We have attained, again, sub + 
milliseconds order performance per uniflability test. 
In addition, this approach exhibited desirable scala- 
bility characteristics. The search time asymptotically 
researches to 450 cycles as the size of classification net- 
work increases. Also, search tinm decreases as average 
fan-out gets larger• Thee are natural advantages of 
using parallel machines• 
SNAP is not only useful for the new and radical 
approach, but also beneficial in speeding up tradi- 
tional NLP systems such as KBMT. We have eval- 
uated the performance to search VLKB which is the 
major knowledge source for the KBMT system. We 
have attained sub-milliseconds order performance per 
a search. Traditionally, on the serial machines, this 
process has been taking a few seconds posing the ma- 
jor thread to performance on the scaled up systems. 
Also, there are many other NLP models (Typed 
Unification Grammar \[Emele and Zajae, 1990\], SNePS 
\[Neal and Shapiro, 1987\], and others) which may ex- 
hibit high performance and desirable scaling property 
on SNAP. 
Currently, we are designing the SNAP-2 reflecting 
various findings made by the research with SNAP-1. 
SNAP-2 will be built upon the state-of-the-art VLSI 
technologies using RISC architecture. At least 32K 
virtual nodes will be supported by each processing ele- 
ment, providing the system with a minimum of 16 mil- 
lion nodes• SNAP-2 will feature nmlti-user supports, 
intelligent I/O, etc. One of the significant features in 
SNAP-2 is the introduction of a programmable marker 
propagation rules. This feature allows users to define 
their own and more sophisticated marker propagation 
rules. 
In summary, we have shown that the SNAP archi- 
tecture can be a useful development platform for high 
performance and large-scale natural language process- 
rag. This has been empirically demonstrated using 
SNAP-1. SNAP-2 is expected to explore opportuni- 
ties of massively parallel natural language processing. 
References 
\[Blelloch, 1986\] Blelloeh, G. E., "CIS: A Massively Par- 
aUel Concurrent Rule-Based System," Proceeding of 
AClT~ DE COLING-92, NA~rEs. 23-28 hOt'q 1992 8 1 8 Paoc. OF COLING-92, NANTES, AUG. 23-28, 1992 
AAAI-86, 1986. 
\[Bzachman and Schmolse, 1985\] tlrachmau, R. J. and 
Schmolze, J. G., "An Overview of The KL-ONE 
Knowledge Representation System," Cognitive Sci- 
ence 9, 171-216, August 1985. 
\[Charniak, 1983\] Charniak, E., "Passing markers: A the- 
ory of contextual influence in language comprehen- 
sion," Cognitive Science, 7(3), 1983. 
\[Carpenter and Pollard, 1991\] Carpenter, B, and Pollard, 
C., "Inclusion, Disjointness and Choice: The Logic 
of Linguistic Classification," Proc. of A cbgJ~ 1991. 
\[EDR, 1988\] Japan Electric Dictionary Research Insti- 
tute, EDR Electric Dictionaries, Technical Report, 
Japan Electric Dictionary Research Institute, 1986. 
\[Emele and Zajac, 1990\] Emele, M. and Zajac, R., 
"Typed Unification Grammars," Proc. of Coting-90, 
1990. 
\[Fahlman, 1979\] Fahhnan, S., NETL: A System for RepT~- 
sen*in 9 and Using Real-World Knowledge, The MIT 
Press, 1979. 
\[Evett, st. aL, 1990\] Evett~ M., ttendler, J., and Spector, 
L., PARKA: Parallel Knowledge Representation on 
the Connection Machine, UMIACS-TR-90-22, Uni- 
versity of Maryland, 1990. 
\[Headier, 1988\] Headier, J., ln~egrating Marker.Passing 
and Problem-Solving, Lawrence Erlbanm Associates, 
1988. 
thirst, 1986\] Hirst, O., Semantic Interpretation and the 
Resolution of Ambiguity, Cambridge University 
Press, Cambridge, 1986. 
Is,cobs, 1991\] Jacobs, P., "Integrating Language and 
Meaning," Sown, J. (Ed.) Principles of Semantic Net- 
works, Morgan Kauflnann, 1991. 
\[Kn.sper~ 1989\] Kasper, R., "Utfilicatlon and Classifica- 
tion: An Experiment in Infonuation-B~sed Parsing," 
Proceedings of the International Workshop on Pars- 
ing Technologies, Pittsburgh, 1989. 
\[Kim and Moldovan, 1999\] Kim, J. and Moldovan, D., 
"Parallel Chmsification for Knowledge Representa- 
tion on SNAP" Proceedings of the 1990 International 
Conference on Parallel Processing, 1990. 
\[Kit.no, 1991\] Kit.no, It., "~DmDialog: An Experimen- 
tal Speech-to-Speech Dialogue Translation System," 
IEEE Computers June, 1991. 
\[Kitano and Higuclfi, 1991a1 Kit.no, H. and Higuchi, T., 
"Massively Parallel Memory-Based Parsing", Pro- 
ceedings of IJCAI-9J, 1991. 
\[Kit.no and Higuclfi, 1991hi Kit.no, H. and Higuchi, T., 
"High Performance Memory-Based Translation on 
IXM2 Massively Parallel Associative Memory Pro- 
cessor", Proceedings of AAAI-91, 1991. 
\[Kit.no st. M., 1991a\] Kit.no, ti., Headier, J., Higuchi, 
T., Moldovan, D., and Waltz, D., "Massively Parallel 
Artificial Intelligence," Proc. of lJCAI-91, 1991. 
\[Kitanoet. al., 1991b\] Kit.no, H., Moldovan, D., and 
Cha, S., "High Performance Natural Language Pro- 
cessing on Semantic Network Array Processor," Prve. 
of IJCAI-91, 1991. 
\[Kit,no, 1990\] Kitano, H., "Parallel Incremental Sentence 
Production for a Model of Simultaneous Interpreta- 
tion," Dale, R., Mellish, C., and Lock, M. (Eds.) 
Current Research in Natural Language Generation t 
Academic Press, London, 1990. 
\[Kit,no st. al., 1989\] Kitnno, H., Tomabechi, H., and 
Levln, L., "Ambiguity Resolution in DmTrans Phm," 
Proceedings of the European Chapter of the Associa- 
tion of Computational Linguistics, 1989. 
\[Lee and Moldovan, 1990\] Lee, W. and Moldovan, D., 
"The Design of a Marker Passing Architecture for 
Knowledge Processing", Proceedings of AAAI-90, 
1990. 
\[Lenat and Guha, 1990\] Lea.t, D.B, and Guha, R.V., 
Building Large K~towledge-Based Systems, Addison- 
Wesley, 1990. 
\[Nagao, 1984\] Nag.o, M., "A b~ramework of. Mechanical 
Translation between Japanese and Engllnh by Anal-. 
ogy Principle," Artificial and Human Intelligence, 
Ehthorn, A. and Banerji, R. (Eds.), Elsevier Scieuce 
Publishers, B.V. 1984. 
\[Neal aud Shapiro, 1987\] Neal, J. and Shapiro, S., 
"Knowledge-Based Parsing," Bole, L., (Ed.) Natural 
Language Parsing Systems, Sptinger-Verlag, 1987. 
\[Goodman and Nirenberg, 1991\] 
Goodman~ K,, and Nirenberg, S. Knowledge-Based 
Machine Translation Project: A Case Study, Morgan 
Kauimann, 1991. 
\[Norvig, 1966\] Norvig, P., Unified Theory of Inference for 
Test Understanding, Ph.D. Thesis, University of Cal- 
ifornia Berkeley, 1986. 
\]Pollard and Sag, 1987\] 
Pollard, C. and Sag, I., b~formation-Based Syntaz 
and Semantics, Vol. L" Fundamentals, CSLI Lecture 
Note Series, Chicago University Press, 1987. 
\[Quilllian, 1968\] Quillian, M. R., "Semantic Memory," Seo 
mastic Information Processing, Minsky, M. (gd.), 
216-270, The MIT press, Cambridge, MA, 1968. 
\[Riesbeck anti Martin, 1985\] Riesbeck, C. and Martin, C., 
"Direct Memory Access Parsing", Yale University 
Report 3S4, 1985. 
\[Riesbeck and Schank, 1989\] Riesbeck, C. and Schank, 
R. Inside Case-Based Reasoning, Lawrence Erlbaum 
Associates, 1989. 
\[Sown, 1991\] S .... J. F. (Ed.), Principles of Semantic 
Networks, Morgan Kaufmann, 1991. 
\[Sown, 1984\] Sown, J. F., ConceptualStrueturen, Reading, 
Addison Wesley, 1984. 
\[St,still and Waltz, 1986\] Stnnfill, C., aud Waltz, D., 
"Toward Memory-Based Reasoning," Communica- 
tion of the ACM, 1986. 
\[Surtdta and lid,, 19911 Sumita, E., and Iida, 1I, "Ex- 
perinmnts and Prospects of Example~Bnsed Machine 
Translation," Proceedings of A CL-91, 1991. 
\[Thinking Machines Corp., 1989\] Thinking Machines 
Corp., Model CM-~ Technical Summary, Technical 
Report TR-89-1~ 1989. 
\[Tomabechi, 1987\] Tomabechi, lI., "Direct Memory Ac- 
cess ~l~anslation', Proceedings of the IJCAI-87, 1987. 
\[Waltz and Pollack, 1985\] Waltz, 1).L. and Pollack, J., 
"Massively Parallel Parsing: A Strongly Interactive 
Model of Natural Language Interpretation" Cognitive 
Science, 9(1): 51-74, 1985. 
\[Wilensky, 1987\] Wilensky, R., "Some Problems and Pro- 
posals for Knowledge Representation", Technical Re- 
port UCB/CSD 87/361, University of California, 
Berkeley, Computer Science Division, 1987. 
\[Zajac, 1989\] Zajac, R., "A Transfer Model Using a Typed 
Feature Structure Rewriting System with Inheri- 
tance," Proc. of A CL-S9, 1989. 
AL~S DE COLING-92, NA~riazs. 23-28 no(rr 1992 8 1 9 Paoc. OF COLING-92. NAN-rE/I, AUG. 23-28. 1992 
