THE LOCALITY PHENOMENON AND PARALLEL PROCESSING OF 
NATURAL LANGUAGE 
E. L. Lozinskii and S. Nirenburg 
Department of Computer Science 
The Hebrew University 
Jerusalem, Israel 
I. Amon~ the various traditions established in computer 
processing of natural language during the twenty-odd years 
of research the understanding thet any such processing is to 
be done sequentially has a special status. 
Even the most advanced natural language processing 
systems employ the sequential mode as a necessary evil, or 
do not even consider it an evil due to the ostensible lack 
of alternatives; thus, for instance, such well-known systems 
as SAM, PAM, ELI /cf. e.g. Schank and Riesbeck, 1981/, PHRAN 
/cf. e.g. Arens, 1981/ or PARSIFAL /see e.g. Marcus, 1979/ 
are all based on sequentionslity. 
The recent advances in the VLSI technology suggest 
%her a re-evaluation of this tradition is in order. Indeed, 
non-sequential "parallel ~ methods start emerging. In the 
field of AI one could mention, for example, Kornfeld's 
(1979, 1981) work in problem solving or the approach of 
HEARSAY-II (see Erman et el., 1980) to speech processing. The 
word parallelism seems even to turn gradually into a current 
"buzz-word" in the AI community. Note that the meaning of 
this word still remains largely loose. Thus, Phillips and 
Hendler (1981) snggest a system of several tss___~k-oriented 
- 186 - 
processors working in parallel. 
2. A different and a more powerful approach to parallel 
processing of natural language is suggested here: in'steed of 
functional distribution we suggest parallel distribution of 
input stream elements; a processor is assigned to avery item 
of input and each such processor is provided with the same 
software package, so that all processes within a certain 
group become equal in status and modus operandi. (Note that 
this also increases the system's reliability, since even in 
the unlikely case of failure of n-1 processes the remaining 
one will accomplish the task by itself, in the sequential 
mod • o~ 
Our approach to parallelism is based on the phenomenon 
of locality. Currently we apply it to constructing a syntact- 
ic parsing system for s subset of English, as e simple case 
of natural language processing. 
3. Let us consider a text as a vector made up of discre- 
te elements wi: 
T = /w0, wl, ..., Wn/. 
Being fed with T a certain Natural Laugue~e Processor (NLP) 
produces a structure of the form S(T) = /v0, Vl, ..., Vm/, 
where vj can be of various nature: words in the object lang- 
uage and/or word~ and symbols in a metalangua~e and/or vari- 
ous kinds of delimiters. 
Let D(Vj) be the minimal subset of T determining v~ in 
the sense that information carried in the elements of th~s 
subset is necessary end sufficient for outputting vj by the 
NLP. Let gj be the index of the leftmost element of D(v.) in 
the string T, and Hi, the index of the rightmost one (e.°g. 
if D(v~) = /w3, ws, w10/ then g~ = 3 and hj = 10). We now de- 
fine the important notion of locality. Locality of an output 
element v;j is l(vS): , - h.i-g~ =~; 
hj hj 
- 187 - 
This function has a number of interesting properties. If an 
output element v~ depends on exactly one element wi, then its 
locality is unity, the highest possible value. On the other 
hand, if a certain v k depends on • large range of input elem- 
ents, then its locality is close to zero. Comparir~ parallel 
and sequential prooessinE we show that the ratio of the time 
necessary to produce an output element in parallel mode to 
that of the sequential mode strictly depends on the locality 
of this element. Moreover, the relative time gain of parallel 
processin8 as regards sequential processing is exactly the 
given element's locality. In other words, the greater the 
~egate locality of elements in a certain text, the more 
benefit there is in its parallel processing. Such is the in- 
trinsic connection between the notion of locality and the per- 
formance of a system based on parallel processing. 
4. The process of implementation starts with finding 
clusters of h~h locality in the text.At this st~e we prove 
the following 
.Proposition. NPs, .VPs and PPs of English are highly local. 
On this basis we proceed to build a system of parallel pars- 
ir~ for ~liah. In its present form the system consists of 
three modules: a morphological and two syntactic ones. The 
result of the first stage is a set of sets of distionary en- 
tries for every input word, which determines the syntactic 
classes to which the input words may in principle belong. A 
~.ammar for each processor et the first syntactic stage of 
analysis is presented as a table which indicates all the cor- 
rect triads of syntactic class members in the subset of Er~l- 
ish We are analyzing. This means that this stnge is devoted 
to finding the states of compatibility between the "neigh- 
bouts" in the input string. It terminates when all the possib- 
le triads have been checked, produces candidates for correct 
parses (if any) and transfers them to the second syntactic 
stage whose task is a/ to carry out all kinds of agreement 
- 188 - 
and completeness tests (e.g. the subject-predicate number 
agreement and the presence of at least one verb in the sent- 
ence, reap.) and b/ to build one or more representations of 
the parse(s) (e.g. a constituent tree and a predicate-role 
structure). This modular framework facilitates the addition 
of new stages to the system, such as one or more semantic 
stages and an inferencir~ mechanism (provided a world is 
defined) • 
The coBunication between separate stages of analysis 
is accomplished globally, and here a secondary parallelism, 
this time the functional one (of. Phillips and Hendler, op. 
cit.)-can be implemented. 

References

A~ens, Y. (1981). Using language and context in the analysis 
of texts. Proceedings of the IJCAT-81. Vancouver,B.C. 

Er~an, L. et el. (1980). The HEAP~AY-XI speech understanding 
system: integrating knowledge to resolve uncertainty. 
ACM Computing Reviews, 12, No. 2. 

Kornfeld, W. a. (1979). Using parallel processing for problem 
solving. MIT AT Lab Memo 561. 

Ko~eld, W. A. (1981). The use of parallelism to i~Dlement 
a heuristic search. Proceedings of the IJCAX-81. Van- 
couver I BoCo 

Marcus, M. (1979)o A theory of syntactic recognition for nat- 
ural ls-,~uage. MIT Press. 

Phillips, B. and J. Hendlar (1981). Controlling parsing by 
passing messages. Proceedings of the Third Annual Con- 
ference of the Cognitive Science Society, Berkeley. 

Schenk, R. C. and C. K. Rieebeck (1981). Inside conputer 
understanding. Lawrence Erlbaum Associates, Hilledale, 
N°J° " 
