ACQUISITION OF SELECTIONAL PATTERNS 
RALPH GRISHMAN and JOHN STERLING 
Computer Science Department 
New York University 
New York, NY 10003, U.S.A. 
1 The Problem 
For most natural language analysis systems, one 
of the major hurdles in porting the system to a 
new domain is the development of an appropri- 
ate set of semantic patterns. Such patterns are 
typically needed to guide syntactic analysis (as 
selectional constraints) and to control the trans- 
lation into a predicate-argument representation. 
As systems are ported to more complex domains, 
the set of patterns grows and the task of accumu- 
lating them manually becomes more formidable. 
There has therefore been increasing interest 
in acquiring such patterns automatically froin a 
sample of text in the domain, through an analysis 
of word co-occurrence patterns either in raw text 
(word sequences) or in parsed text. We briefly 
review some of this work later in the article. We 
have been specificaily concerned about the prac- 
ticality of using such techniques in place of man- 
ual encoding to develop the selectional patterns 
for new domains. In the experiments reported 
here, we have therefore been particularly con- 
cerned with the evaluation of our automatically 
generated patterns, in terms of their complete- 
hess and accuracy and in terms of their efficacy 
in performing selection during parsing. 
2 Patterns and Word Classes 
In principle, the semantic patterns could be 
stated in terms of individual words - this verb 
can meaningfully occur with this subject, etc. In 
practice, however, this would produce an unman- 
ageable number of patterns for even a small do- 
main. We therefore need to define semantic word 
classes for the domain and state our patterns in 
terms of these classes. 
Ideally, then, a discovery proeednre for seman- 
tic patterns would acquire both the word classes 
and the patterns from an analysis of the word 
co-occurrence patterns. In order to simplify the 
task, however, while we are exploring different 
strategies, we have divided it into separate tasks, 
that of acquiring word classes and that of ac- 
quiring semantic patterns (given a set of word 
classes). We have previously described \[1\] some 
experiments in which the principal word classes 
for a sublanguge were obtained through the clus- 
tering of words based on the contexts in which 
they occurred, and we expect to renew such ex- 
periments using the larger corpora now available. 
However, the experiments we report below are 
limited to the acquisition of semantic patterns 
given a set of manually prepared word classes. 
3 Pattern Acquisition 
The basic mechanism of pattern acquisition is 
straightforward. A sample of text in a new do- 
main is parsed using a broad-coverage grammar 
(but without any semantic constraints). The re- 
sulting parse trees are then transformed into a 
regularized syntactic structm'e (similar to the f- 
structure of Lexical-Fnnctional Grammar). This 
regularization in particular reduces all different 
clausM forms (active, passive, questions, extra- 
posed forms, relative clauses, reduced relatives, 
etc.) into a uniform structure with the 'logical' 
subject and object explicitly marked. For exam- 
ple, the sentence 
Fred ate fresh cheese from France. 
would produce the regularized syntactic struc- 
ture 
(s eat (subject (np Fred)) 
(object (np cheese (a-pos fresh) 
(from (np France))))) 
We then extract from this regularized structure 
a series of triples of the form 
AcrEs DE COLING-92. NAMES. 23-28 hot;r 1992 6 5 8 PROC. OF COLING-92, NANTES, AUG. 23-28, 1992 
head syntactic-function value 
where - if the value is another NP or S - only 
the head is recorded. For example, for the above 
sentence we would get tile triples 
eat subject Fred 
eat object cheese 
cheese a-pos fresh 
cheese from bYance 
Finally, we generalize these triples by replacing 
words by word classes. We had previously pre- 
pared, by a purely manual analysis of the corpus, 
a hierarchy of word classes and a set of semantic 
patterns for the corpus we were using. From this 
hierarchy we identified the classes which were 
most frequently referred to in the nlanually pre- 
pared patterns. The generalization process re- 
places a word by the most specific class to which 
it belongs (since we have a hierarchy with uested 
classes, a word will typically belong to several 
classes). As we cxplain in our experiment sec- 
tion below, we made some runs generalizing just 
the value and others generalizing both tlle head 
and the value. 
As we process the corl)us , we kee t) a count of 
the frequency of each head-function wdue triple. 
In addition, we keep separate counts of the num- 
ber of times each word appears as a head, and 
the number of times eacll head-fitnction pair al)- 
pears (independent of value). 
4 Coping with Multiple Parses 
The procedure described above is sufficient if 
we are able to obtain ttlc correct parse for eacll 
sentence, llowever, if we are porting to a new 
domain and have no semantic constraints, we 
lnust rely entirely upon syntactic constraints and 
so will be confronted with a large number of 
incorrect parses for each sentence, along with 
(hopefully) the correct one. We have exl)eri- 
mented with several approaches to dealing with 
this problem: 
1. If a sentence has N parses, we can generate 
triples front all the parses and tllen include 
each triple with a weight of 1/N. 
2. We can generate a stochastic grammar 
through unsupervised training on a portion 
of the corpus \[2\]. We can then parse the cor- 
pus with this stochastic grammar at,l take 
only the most probable parse for each sen- 
tence. \]:or sentences which still generated 
N > 1 equally-probable parses, we would 
use a 1/N weight ;us before. 
3. In place of a 1/N weighting, we can re- 
tine the weights for alternative parse trees 
nsing an iterative procedure analogous to 
the inside-outside algorithm \[3\]. We he- 
gin by generating all parses, as in approach 
1. Then, based on the counts obtained ini- 
tially (using 1/N weighting), we can com- 
pute the probability for the various triples 
attd from these tim probabilities of the al- 
ternative parse trees. We can then repeat 
the process, recomputing the counts with 
weightings based on these probabilities. 
All of these approaches rely on the expectation 
that correct patterns arising from correct parses 
will occur repeatedly, while the distribution of 
incorrect patterns from incorrect parses will be 
more scattered, and so over a sufficiently large 
corpus- we cat, distinguish correct from incor- 
rect patterns on the basis of frequency. 
5 Evaluation Methods 
~\['o gather patterns, we analyzed a series of arti- 
cles on terrorism which were obtained from tim 
Foreign llroadcast lnh)rmation Service and used 
as the development era'pus for the Third Message 
Understanding Confiwence (held in San Diego, 
CA, May 1991) \[4\]. l'br pattern collection, we 
used 1000 such articles with a total of :14,196 
sentences and 330,769 wor(ls. Not all sentences 
parsed, both because of limitations in our gram- 
mar and becanse we inlpose a limit on the search 
which the parser can perform for each sentence. 
Within these limits, we were able to parse a total 
of 7,455 sentencesJ 
The most clearly definable function of tbe 
triples we collect is to act as a selectional con- 
straint: to differentiate between meaningfld and 
meaningless triples in new text, and thus identify 
the correct attalysls. 
We used two methods to evaluate the effec- 
tiveness of tile triples we generated. The first 
IF or these runs we disabled several heuristics in our 
systelll which increase tile nulnbef of sentences which can 
be parsed at some cost m the average quality of parses; 
hence the relatively low percentage of sentences which ob- 
tained parses. 
ACRES DE COLING-92, NANrES, 23-28 Ao(rr 1992 6 5 9 PROC, OF COLING-92. NANrrEs, AUO. 23-28, 1992 
method involved a comparison with manually- 
classified triples. We took 10 articles (not in the 
training corpus), generated all parses, and pro- 
duced the triples from each parse. These triples 
were stated in terms of words, and were not 
generalized to word classes. We classified each 
triple as semantically valid or invalid (a triple 
was counted as valid if we believed that this pair 
of words could meaningfully occur in this rela- 
tionship, even if this was not the intended rela- 
tionship in this particular text). This produced a 
test set containing a total of 1169 distinct triples, 
of which 716 were valid and 453 were invalid. 
We then established a threshold T for the 
weighted triples counts in our training set, and 
defined 
v+ number of triples in test set which were clas- 
sified as valid and which appeared in train- 
ing set with count > T 
v_ number of triples in test set which were clas- 
sified ms valid and which appeared in train- 
ing set with count < T 
i+ number of triples in test set which were classi- 
fied ms invalid and which appeared in train- 
ing set with count > T 
i_ number of triples in test set which were classi- 
fied as invalid and which appeared in train- 
ing set with count < T 
and then defined 
recall - v+ v++v_ 
precision - v+ v+ + i+ 
i+ error rate - 
i++i_ 
By varying tim threshold, we can plot graphs 
of recall vs. precision or recall vs. error-rate. 
These plots can then be compared among differ- 
ent strategies for collecting triples and for gen- 
eralizing triples. The precision figures are some- 
what misleading because of the relatively small 
number of invalid triples in the test set: since 
only 39% of the triples are invalid, a filter which 
accepted all the triples in the test set would still 
be accounted as having 61% precision, We have 
therefore used the error rate in the figures below 
(plotting recall against l~rror-rate). 
The second evaluation method involves the 
use of the triples in selection and a comparison 
of the parses produced against a set of known 
correct parses. In this case the known correct 
parses were prepared manually by the Univer- 
sity of Pennsylvania as part of their "'Free Bank" 
project. For this evaluation, we used a set of 317 
sentences, again distinct from the training set, 
In comparing the parser output against the stan- 
dard trees, we measured the degree to which the 
tree structures coincide, stated as recall, preci- 
sion, and number of crossings. These measures 
have been defined in earlier papers \[5,6,7\]. 
6 Results 
Our first set of experiments were conducted to 
compare three methods of coping with multiple 
parses. These methods, as described in section 4, 
are (1) generating all N parses of a sentence, and 
weighting each by l/N; (2)selecting the N most 
likely parses as determined by a stochastic gram- 
mar, and weighting those each by 1/N; (3) gen- 
erating all parses, but assigning weights to alter- 
native parses using a form of the inside-outside 
procedure. These experiments were conducted 
using a smaller training set, a set of 727 sen- 
tences drawn from 90 articles. We generated a 
set of triples using each of the three methods and 
then evaluated them against our hand-classified 
triples, as described in section 5. We show in 
Figure 1 the threshold vs. recM1 curves for the 
three methods; in Figure 2 the recall vs. 1-error 
rate curves. 
These experiments showed only very small dif- 
ferences between the three methods (the inside- 
outside method showed slightly better accuracy 
at some levels of recall). Based on this, we 
decided to use method 2 (statistical grammar) 
for subsequent experiments. Other ttfings being 
equal, method 2 hms the virtue of generating far 
fewer parses (an average of 1.5 per sentence, vs. 
37 per sentence when all parses are produced), 
and hence a far smaller file of regularized parses 
(about 10 MB for our entire training corpus of 
1000 articles, vs. somewhat over 200 MB which 
would have been required if all parses were gener- 
ated). Using method 2, therefore, we generated 
the triples for our 1000-article training corpus. 
Our second series of experiments compared 
three different ways of accumulating data from 
the triples: 
ACTES DE COLING-92, NANTES. 23-28 hot~r 1992 6 6 0 PRoc. OF COLING-92, NANTES, AUG. 23-28, 1992 
0.40 - 
, 00o 
0.30 - * *~'~ 
recall 0.20 - '~ 
O 
O 
0.10 - '~l~ 
0.00- --, ....... , , ~ ..... *, , -'~,,,I 
0.1 1 10 100 
threshold 
Figure 1: Comparison of methods for dealing 
with multiple parses in pattern collection, us- 
ing training corpus of 90 articles. Threshokl 
vs. recall for o = all parses; o = all parses + 
inside-outside; • = most t)robable parses from 
stochastic grammar. 
1.00 -**** 
ooo 
0.95 - me 
oO 
\] • 
error 0.90- rate 
0.85 - 
o o5 
moo 
o 
% 
0.80" ' ', , i --7 
0.00 0.10 0.20 0.30 0.40 
recall 
Figure 2: Comparison of methods for dealing 
with multiple parses in pattern collection, using 
training corpus of 90 articles. RecM1 vs. l-er 
rot rate for o = all parses; o = all parses + 
inside-outside; • = most probable parses from 
stochastic grammar. 
1.00 
• ~mmo 0.80 .m 
o. o \ 
recall % 
0.40 o~O 0 
(I.20 
0.00 ....... ~ ....... 
0.1 l 10 100 
threshold 
Figure 3: Comparison of pattern generalization 
techniques, using training corpus of 1000 arti- 
cles. Threshold vs. recall for o = triples without 
generalized heads; o = triples with generalized 
heads; • = 1)airs. 
1. generalizing the value in a head-flmction- 
value triple to a word class, but not gen- 
eralizing tile head 
2. generalizing hoth the value and the head 
3. ignoring the value field entirely in a 
head-function-value triple, ,~n(l accumulat- 
ing counts of head-fimction pairs (with no 
generalization applied to the head); a match 
with the hand-marked triples is therefore 
recorded if the head and flmction fields 
match 
Again, we evaluated the patterns produced by 
each method against tile hand-marked triples. 
Figure 3 shows the threshohl vs. recall curves 
for each method; Figure 4 the recM1 vs. 1-error 
rate curves. Figure 3 indicates that using pairs 
yields the highest recall for a given threshold, 
triples with generalized \]leads an intermediate 
value, and triples without generalized heads the 
lowest recall. The error rate vs. recall curves 
of ligure 4 do not show a great difference be- 
tween mcdLods, but they do indicate ttlat, over 
tile range of recalls for which they overlap, using 
triples without generalized heads l)roduces the 
lowest error rate. 
Finally, we conducted a series of experiments 
to compare the effectiveness of the triples in se- 
lecting the correct parse, in effect, the selection 
procedure works as follows, l'br each sentence in 
the test corpus, the system generates all possible 
AcrEs DE COLING-92, NANTES, 23-28 Ao~r 1992 6 6 1 PROC. OF COLING-92. Nx~,~r~s. AUG. 23-28. 1992 
1 - 
error rate 
1.00 - 
0.80 - 
0.60 
0.40 
0.20 
0 o 
| 
| 
| 
I I ' I I I I ' I" "1 I" '1 
L00 0.20 0.40 0.60 0.80 \].00 
recall 
Figure 4: Comparison of pattern generalization 
techniques, using training corpus of 1000 articles. 
Recall vs. 1-error rate for o = triples without 
generalized beads; o = triples with generalized 
heads; • = pairs. 
parses and then generates a set of triples from 
each parse. Each triple is assigned a score; the 
score for the parse is the product of the scores 
of the triples obtained from the parse (the use 
of products is consistent with the idea that the 
score for a triple to some degree reflects the prob- 
ability that this triple is semantically valid). The 
parse or parses with the highest total score are 
then selected for evaluation. 
We tested three approaches to assigning a 
score to a triple: 
1. We used the frequency of head-function- 
value triples relative to the frequency of the 
head as an estimate of the probability that 
this head would appear with this function- 
value combination. We used the "expected 
likelihood estimate" \[8\] in order to assure 
that triples which do not appear in the train- 
ing corpus are still assigned non-zero proba- 
bility; this simple estimator adds 1/2 to each 
observed frequency: 
freq. of triple + 0.5 score = 
freq. of head + 0.5 
2. We applied a threshold to our set of col- 
lected triples: if a triple appeared with a 
frequency above the threshold it was as- 
signed one score; if at or below the thresh- 
old, a lower score. We selected a threshold 
of 0.9, so that any triple which appeared 
unambiguously in at least one sentence of 
the training corpus was included. For our 
scores, we used the results of our previ- 
ous set of experiments. These experiments 
showed that at a threshold of 0.9, 82% of 
the triples above the threshold were seman- 
tically valid, while 47% of the triples below 
the threshold were valid3 Thus we used 
score = 0.82 if freq. of triple > 0.9 
0.47 if freq. of triple < 0.9 
We expanded on method 2 by using both 
triples and pairs information. To assign 
a score to a head-function-value triple, we 
first ascertain whether this triple appears 
with frequency > T in the collected pat- 
terns; if so, we assign a high score to the 
triple. If not, we determine whether the 
head-function pair appears with frequency 
> T in the collected patterns. If so, we 
assign an intermediate score to the triple; 
if not, we assign a low score to the triple. 
Again, we chose a threshold of 0.9 for both 
triples and pairs. Our earlier experiments 
indicated that, of those head-function-value 
triples for which the triple was below the 
threshold for triples frequency but the head- 
function pair was above the threshold for 
pair frequency, 52% were semanticaily valid. 
Of those for which the head-function pair 
was below the threshold for pair frequency, 
40% were semantically valid. Thus we used 
score = 0.82 if freq. of triple > 0.9, else 
0.52 if freq. of pair > 0.9, else 
0.40 if freq. of pair < 0.9 
Using these three scoring flmctions for selec- 
tion, we parsed our test set of sentences and then 
scored the resulting parses against our "standard 
parses". As a further comparison, we also parsed 
the same set using selectional constraints which 
had been previously manuMly prepared for this 
domain. The parses were scored against the stan- 
dard in terms of average recall, precision, and 
number of crossings; the results are shown in Ta- 
ble 1. 3 A better match to the correct parses 
2"Fhe actual value of the scores only matters in cases 
where one parse generates more triples than another. 
3These averages are calculated only over the subset of 
test sentences which yielded a parse with our granunar 
within the edge limit alloted. 
ACTI':S DE COLING-92, NANTES, 23-28 AOUT 1992 6 6 2 PROC. OF COL1NG-92, NAMES, AUG. 23-28, 1992 
selection strategy crossings rec',di precision 
1. frequency-based 2.00 75.70 71.86 
2. triples-threshold 2.17 73.57 70.22 
3. triples-and-pairs ° 2-09 74:33 70.94- 
3. hand-generated 2.04 "t4.34 i 
Table 1: A comparison of the effect of different 
selection strategies on the quality of parses gen- 
erated. 
is reflected in higher recall and precision and 
lower number of crossings. These results indi- 
cate that the frequency-based scores performed 
better than either the threshold-ha.qed scores or 
the manually-prepared selection. 
7 Related Work 
At NYU we have long been interested in the pos- 
sibilities of automatically acquiring sublanguage 
(semantic) word classes and patterns from text 
corpora. In 1975 we reported on experiments 
-- using a few hundred manuMly prepared reg- 
ularized parses --- for clustering words based on 
their co-occurrence patterns and thus generat- 
ing the principal sublanguage word classes for a 
domain \[1\]. In the early 1980's we performed 
experiments, again with relatively small corpora 
and machine-generated (but manually selected) 
parses, for collecting snblanguage patterns, simi- 
lar to the work reported here \[9\]. By studying the 
growth curves of size of text sample vs. number of 
patterns, we attempted to estimate at that time 
the completeness of the subtanguage patterns we 
obtained. 
More recently there has been a surge of in- 
terest in such corpus-based studies of lexicai co 
occurrence patterns (e.g., \[1{},11,12,13\]). The re- 
cent volume edited by Zernik \[14\] reviews many 
of these efforts. We mention only two of these 
here, one seeking a similar range of patterns, the 
other using several ewduation methods. 
Velardi et al. \[11\] are using co-occurence data 
to build a "semantic lexicon" with information 
about the conceptual classes of the arguments 
and modifiers of lexical items. This informa- 
tlon is closely related to our selectional patterns, 
although the function'a\] relations are semantic 
or conceptual whereas ours are syntactic. They 
use manually-encoded coarse-grained selectional 
constraints to limit the patterns which are gen- 
erated. No evaluation results are yet reported. 
IIindle aml Rooth \[10\] h~ve used co-occurrence 
data to determine whether prepositional phrases 
should be attached to a preceding noun or verb. 
Unambiguous cases in the corpus are identified 
first; co-occurrence statistics based on these are 
then used iteratively to resolve ambiguous cases. 
A detailed evaluation of the predictive power of 
the resulting p~tterns is provided, comparing the 
patterns against human judgements over a set of 
1909 sentences, aud analyzing the error rate in 
terms of the type of verh and noun association. 
8 Conclusion 
We have described two different approaches 
to evahtating automatically collected selectional 
patterns: by comparison to a set of manually- 
classified patterns and in terms of their effective- 
hess in selecting correct parses. We have shown 
that, without any manual selection of the parses 
or patterns ilt our trMning set, we are able to 
obtain selectioual p~tterns of quite satisfactory 
recall and precision, and which perform better 
than a set of manual selectional patterns in se~ 
lecting correct parses. We are not aware of any 
comparable etlorts to evaluate a hdl range of au- 
tomatically acquired selectional patterns. 
Further studies are clearly needed, particularly 
of the best way in which the collected triples can 
be used for selection. The expected likelihood 
estimator is quite crude and more robust estima- 
tors should be tried, particularly given the sparse 
nature of tim data. We should experiment with 
better ways of combining of triples and pairs data 
to give estimates of semantic validity. Finally, we 
need to explore ways of combining these auto- 
tactically collected patterns with manually gen- 
erated selectional patterns, which will probably 
have narrower coverage but may be more precise 
and complete for the w~rbs covered. 
9 .Acknowledgements 
Thi~ report is based upon work supported by 
the Defense Advanced Research Projects Agency 
under Grant N00014-9O-J-1851 from the Office 
of Naval Research and by the National Science 
l:oun(lation under Grant 11H-89-02304. 
AcrEs DE COLING-92. NANTES, 23-28 ^oLrr 1992 6 6 3 |)ROC. or COLING-92. NAN'rE.S. Atro. 2.3-28, 1992 

References 

\[1\] Lynette ttirschman, Ralph Grishman, and 
Naomi Sager. Grammatically-based auto- 
matic word class formation. Information 
Processing and Management, 11(1/2):39- 
57, 1975. 

\[2\] Mahesh Chitrao and Ralph Grishman. Sta- 
tistical parsing of messages. In Proceedings 
of the Speech and Natural Language Work- 
shop, pages 263--266, Hidden Valley, PA, 
June 1990. Morgan Kaufmann. 

\[3\] J. K. Baker. Trainable grammars for speech 
recognition. In D. H. Klatt and J. J. Wolf, 
editors, Speech Communication Papers for 
the 97th Meeting of the Acoustic Society of 
America, 1979. 

\[4\] Beth Sundheim. Third message under- 
standing evaluation and conference (MUC- 
3): t'hase 1 statns report. In Proceedings of 
the Speech and Natural Language Workshop, 
pages 301-305, Pacific Grove, CA, 1,'ebruary 
1991. Morgan Kaufmann. 

\[5\] Ezra Black, Steven Abney, Dan Flickenger, 
Claudia Gdaniec, Ralph Grishman, Philip 
tlarrison, Donald Hindle, Robert Ingria, 
Fred Jelinek, Judith Klavans, Mark Liber- 
man, Mitch Marcus, Salim Roukos, Beatrice 
Santorini, and Tomek Strzalkowski. A pro- 
cedure for quantitatively comparing the syn- 
tactic coverage of English. In Proceedings of 
the Speech and Natural Language Workshop, 
pages 306-311, Pacitic Grove, CA, February 
1991. Morgan Kaufinann. 

\[6\] Philip tlarrison, Steven Abney, Ezr,~ Black, 
Dan Flickinger, Claudia Gdaniec, Ralph 
Grishman, Donald ltindle, Robert lngria, 
Mitch Marcus, Beatrice Santorini, and 
Toiuek Strzalkowski. Evaluating syntax 
performance of parser/grammars. In Pra- 
ceedings of the Natural Language Process- 
ing Systems Evaluation Workshop, Berke- 
ley, CA, June 1991. To be published as a 
Rome Laboratory Technical Report. 

\[7\] Ralph Grishman, Catherine Macleod, and 
John Sterling. Evaluating parsing strate- 
gies using standardized parse files. In Proe. 
Third Conf. on Applied Natural Language 
Processing, Treuto, Italy, April 1992. 

\[8\] William Gale and Kenneth Church. Poor 
estimates of context are worse than none. 
In Proceedings of the Speech and Natural 
Language Workshop, pages 283 287, tlidden 
Valley, PA, June 1990. Morgan Kaufmann. 

\[9\] R. Grishman, L. Hirschman, and N.T. 
Nhan. Discovery procedures for sub- 
language selectional patterns: Initial ex- 
periments. Computational Linguistics, 
12(3):205-16, 1986. 

\[10\] Donald IIindle and Mats Rooth. Structural 
ambiguity and lexical relations. In P~aceed~ 
ings of the 29th Annual Meeting of the Assn. 
for Computational Linguistics, pages 229 
236, Berkeley, CA, June 1991. 

\[11\] Paota Velardi, Maria Teresa Pazienza, and 
Michela Fasolo. How to encode semantic 
knowledge: A method for meaning repre- 
sentation and computer-aided acquisition. 
Computational Linguistics, 17(2):153-170, 
1991. 

\[12\] Frank Smadja. From n-grams to colloca- 
tions: An evaluation of xtract. In Proceed- 
ings of the 29th Annual Meeting of the Assn. 
for Computational Linguistics, pages 279-- 
284, Berkeley, CA, June 1991. 

\[13\] Nicolette Calzolari and Remo Bindi. Acqui- 
sition of lexical information from a large tex- 
tual italian corpus. In Proc. 13th Int'l Conf. 
Computational Linguistics (COLING-90), 
pages 54-59, Iletsinki, Finland, August 
1990. 

\[ld\] Uri Zernik, editor. Lexical Acquisition: Ex- 
ploiting On-Line Resou~ves to Build a Lex- 
icon. Lawrence Erlbaum Assoc., tlillsdale, 
N J, 1991. 
