AN EXPERIMENT ON " " " L b,\.RNINC APPROPRIATE 
SELECTIONAL RESTRICTIONS I?I{OM A PARSED CORPUS 
1,'r~mccsc l\[ibas l?ramis* 
Departament de Llenguatges i Sist;emes Inform~ttics, Universitat; PolitScnica de Catalunya 
Pau Gargallo 5, 08082 Barcelona, SPAIN. eqnaih ribasQlsi.upc.es 
Abstract 
We present a methodology to extract Selectional 
Restrictions at a variable level of abstraction from 
phrasally analyzed corpora. The method relays ia the 
use of a wide-coverage noun taxonomy and a statis- 
tical measure of the co-occurrence of linguistic items. 
Some experimental results about the performance of 
the method are provided. 
Keywords: large text corpora, computational 
lexicons 
1 INTI~ODUCTION 
These last years there has been a common agreement 
in the natural language processing research community 
on the importance of having an extensive coverage of 
the surface lexical semantics of the domain to work 
with, (specially, typical contexts of use). 'iFhis knowl- 
edge may be expressed at different levels of abstraction 
depending on the phenomena involved: selecLional re~ 
strictions (SR~), lexical preferences, eel-locations, etc. 
We are specially interested on SIhs, which can be ex-. 
pressed as semantic type constraints t/tat a word se~se 
imposes on the words with which it combines in the 
process of semantic interpretation. SRs nmst iachlcle 
information on the syntactic position of the words thai; 
are being restricted semantically. For instance, one of 
the senses of the verb drink restricts its subject ~o be 
an animal and its object to be a liq'uid. 
SILs may help a parser to prefer some parses among 
several grammatical ones \[WFB90\]. Furthermore, S\]{s 
may help the parser when deciding tilt s(-mantic role 
played by a syntactic complement. Lexicography is 
also interested in the acquisition of SlCs. On the one 
hand, SRs are an interesting inform~xtion to be im:Imied 
in dictionaries (defining in co~dezt approach). Oa th(; 
other band, ;m \[ClI90\] remark, the e\[lbrt involved ht all- 
*This research has been supported by a grant conceded by the 
Generalitat de Catahmya, 91-1)OGC-1,191. Much of the work re- 
ported here wa-n carried out, during a visit, at. the Cl)tllptt\[el' l,ab- 
oratory, University of Cambridge. 11 am grateful t,o Ted Briscoe 
and t\[oraclo Rodriguez by their vMuab\[e eo~tHlleltt.s. 
alyzing and cl~ussifying all tile linguistic material pro- 
vided by concordances of use of a word can be ex- 
tremely labor-intensiw. ~. If it was possible to represent 
roughly the Sits of the word being studied, it could be 
possible to clmssify roughly the concordances automati- 
cally in the different word uses before the lexicographer 
analysis. 
The possible sources of Sits are: introspection by 
lexicographers, machine-readable dictionaries, ~nd on-- 
line corpora. 'l'he main advantage of the latter is that 
they provide experimental evidence of words uses. tl.e-- 
cently, several approaches on acquiring different kinds 
of lexical information from corpora have been devel- 
oped \[BPV92, CG\[III91, CH90, Res92\]. This paper 
is interested in exploring the amenability of using a 
method f(~r extracting SI{~ from textual data, in the 
line of these works. The aim of the proposed tech- 
nique is to learn the Sl~,s that a word is imposing, from 
the analysis of the examples of use of that word con- 
taim'd in the corpus. An illustration of such a learn- 
ing is shown in Figure l, where the system, depart- 
ing from the three examples of use, and knowing that 
prosec'utor, buyer attd lawmaker are nouns belonging 
to the semantic class <pera-m~, individual >, and that 
i~zdictme~d, assura~zce and legislation are members of 
< legal_in.strttmeuZ >, should induce that the verb see/.' 
imposes SILs that constraint the subject to be a nmm-. 
bet of the semantic type <peraon, individval>, and 
the object to be a kind of < legaLiustrurnent >. Con- 
eluding, the system should extract for each word (with 
contplemeut,..~) having enough number occurrences of 
use in the corpus and for each of its syntactic eomple- 
melttS, a li:;t of the alternative Sl~s that this word is 
imposing. 
In order to detect the SRs that a word imposes 
in il;s coutext hy means of statistical techniques two 
distiuct approaches haw~, been proposed: word-based 
\[CC, HIIgl\], and class..based \[Bpv92, ~tes92\]. Word-. 
based al~proach infers SRs as the collection of words 
that co-occur significantly in the syntactic context of 
the studi,'.d word. The clmss--based techniques gather 
the; dillhrene nouns by means of semantic cl,'uqses. The 
advantages of the latter are clear. On the one hand, 
sl.atist.ically meaningful data can be gathered From (tel-- 
769 
Figure 1: Example of the acquisition of Slgs for the 
verb seek from three examples of use 
• Three examples of use 
prosecutors may soon seek an indictment on rack- 
eteering and securities fraud charges. 
In the recent past, bond buyers didn't seek such 
assurance. 
Some lawmakers may seek legislation to limit 
overly restrictive insurance policies. 
• The extracted SILs 
(seek, subject, <person, individual>) 
(seek, object, < legal_instrument >) 
atively) small corpora,and not only for the most fre- 
quent words. On the other hand, SRs are generalized 
to new examples not present in the training set. Fi- 
nMly, the acquired SRs are more independent of the 
lexical choices made in the training corpus. 
We have developed and implemented a method for 
automatically extracting class-based SRs from on-line 
corpora. In section 2 we describe it while discussing 
other approaches. In section 3 we analyze some data 
about the performance of an experiment run in a Unix 
machine, on a corpus of 800,000 words. Finally, in sec- 
tion 4 we discuss the performance achieved, and sug- 
gest further refinements of the technique in order to 
solve some remaining problems. 
2 THE METHOD OF ACQUISITION 
SRs have been used to express semantic constraints 
holding in different syntactic and functional configu- 
rations. However, in this paper we focus only in se- 
lectional restrictions holding between verbs and their 
complements. The method can be easily exported to 
other configurations. We won't distinguish the SiTs im- 
posed by verbs on arguments and adjuncts. We believe 
that few adjuncts are going to provide enough evidence 
in the corpus for creating SRs. In the following para- 
graphs we describe the functional specification of the 
system. 
Training set The input to the learning process is 
a list of co-occurrence triples codifying the co- 
occurrence of verbs and complement heads in the 
corpus: (verb, syntaclic relationship, noun). Verb 
and noun are the lemrnas of the inflected forms ap- 
pearing in text. Syntactic relationship codes the 
kind of complement: 0subject, I object , or prepo- 
sition in case it is a PP. A method to draw the 
co-occurrence triples from corpus is proposed in 
subsection 2.1. 
Output The result of the learning process is a set 
of syntactic SPas, (verb, syntactic relationship, se- 
mantic class}. Semantic classes are represented 
extensionally as sets of nouns. SRa are only ac- 
quired if there are enough cases in the corpus as 
to gather statistical evidence. As long as distinct 
uses of the same verb can have different SRs, we 
permit to extract more than one class for the same 
syntactic position. Nevertheless, they must be 
umtually disjoint, i.e. not related by hyperonymy. 
Previous knowledge used In the process of learn- 
ing SRs, the system needs to know how words are 
clustered in semantic classes, and how semantic 
classes are hierarchically organized. Ambiguous 
words must be represented ms having different hy- 
peronym classes. In subsection 2.2 we defend the 
use of a b'road-coverage taxonomy. 
Learning process The computational process is di- 
vided in three stages: (1) Guessing the possible 
semantic classes, i.e. creation of the space of can- 
didates. In principle, all the hyperonyms (at all 
levels) of the nouns appearing in the training set 
are candidates. (2) Evaluation of the appropriate~ 
hess of the candidates. In order to compare the 
different candidates, art statistical measure sum- 
ma.rizing the relevance of the occurrence of each of 
the candidate classes is used. (3) Selection of the 
most appropriate subset of the candidate space to 
convey the SILs, taking into account that the final 
classes must be mutually disjoint. While in sub- 
section 2.3 an statistical measure to flflfill stage 2 
is presented, stages 1 and 3 are discussed in 2.4 
thoroughly. 
2.1 Extracting Co-occurrence Triples 
In any process of learning from examples the accuracy 
of the training set is the base for the system to make 
correct predictions. In our case, where the semantic 
classes are hypothesized not univoquely from the ex~ 
staples, accuracy becomes fundamental. 
Different approaches to obtain lexical co-occurren- 
ces have been proposed in the literature \[BPV92, 
CGHH91, CH90\]. These approaches seem inappro- 
priate for tackling our needs, either because they de- 
tect only local co-occurrences\[CGHtI9i, CtI90\], or be- 
cause they extract many spurious co-occurrence triples 
\[BPV92, Clt90\]. On the one hand, our system in- 
tends to learn SRs on any kind of verb's complements. 
On the other hand, the fact that these approaches ex- 
tract co-occurrences without reliability on being verb- 
complements violates accuracy requirements. 
However, if the co-occurrences were extracted from 
a corpus annotated with structural syntactic informa- 
tion (i.e., part of speech and "skeletal" trees), the re- 
sults would have considerably higher degrees of accu- 
770 
racy and representativity. In this way, it would be easy 
to detect all tile relationships between verb and con> 
plements, and few non-related co-occurrences would be 
extracted. 'rile most serions objection to this approach 
is that the task of producing syntactic analyzed cor- 
pora is very expensive. Nevertheless, lately there has 
been a growing interest to produce skeletally analyzed 
corpora 1 
A parser, with some simple heuristics, wonI(1 be 
enough to meet the requirements of representativeness 
and accuracy introduced above• On the other band, it 
could be useful to represent the co-occurrence triples 
as holding between lemmas, in order to gather ~ much 
evidence as possible. A simple morphological analyzer 
that could get the lemma for a big percentage of tile 
words appearing in the corpus would suffice. 
2.2 Semantic Knowledge Used 
Of the two class-based approaches presented in section 
1, \[Res92\]'s technique uses a wide-coverage semantic 
taxonomy , whereas \[BPV92\] consists in hand-tagging 
with a fixed set of semantic labels . The advantages 
and drawbacks of both approaches are diverse. On 
the one hand, in \[BPV92\] approach, semantic classea 
relevant to the domain are chosen, and consequently, 
the adjustment of the classes to the corpus is quite nice. 
Nevertheless, \[I'~es92\]'s system is less constr~ined and is 
able to induce a most appropriate level for the SRs. On 
the other hand, while \[BPV92\] implies hand-coding all 
the relevant words with semantic tags, \[\[{.es92\] needs a 
broad semantic taxonomy. IIowever, there is already 
an available taxonomy, WordNet 2. We take ires92\] 
approach because of the better results obtained, and 
the lower cost involved. 
2.3 Class appropriateness: the Associ- 
ation Score 
When trying to choose a measure of the appropriate-. 
hess of a semantic class, we have to consider the fea- 
tures of the problem: (1) robustness in front of noise, 
and (2) conservatism in order to be able to general- 
ize only front positive examples, without having the 
tendency to over-generalize. 
Several statistical measures that accomplish these 
requirements have been proposed in the literature 
\[BPV92, CGIItI91, l{es92\]. We adopt \[Res92\]'s ap- 
proach, which qnantifies tile statistical association 
XFor instance, Penn Treebank Corpus, which is belug col- 
lected and ana|yzed by the University of Penl~sylwmia (sec 
\[MSM93\]). The material is available on request, from the l,in- 
guistic Data Consortium, (email) ldc@unagi.cis.upenn.ed, tt 
=WordNet is a lexieal datab.~e developed with psycholitt- 
guistlc aims. it represents lexiea\[ scnlantics infl)t'Inatlon about 
nouns, verbs, adjectives and adverbs such as hypevonyms, 
meronyms, ... it presently contains information on about 83,000 
lemtn,'~q. See \[MBF + 90\] 
between verbs and classes of nouns from their co- 
occurrenee. IIowever we adapt it taking into account 
tile syntactic position of the relationship. Let 
v = {,, ..... ~,,}, :¢-- {,u ..... ,,,,,}, 
a = {0,1,,0, ~t .... }, ~,.t c = {~1¢ c N} 
be tim sets of all verbs, nouns, syntactic positions, 
and possible noun classes, respectively. Given v E V, 
s E {'; and c G C, Association Score, Assoc, between v 
and e in a syntactic position s is defined to be 
A~.~oc(~, ~., ~) =_ P(e/,,, ~)~(~; ~/~) 
= 1"(cI,,, s) log= e(,,, c/s) 
Where conditional probabilities are estimated by 
counting the number of observations of tile joint event 
and dividing by the frequency of the given event, e.g 
~,,ec eo,.~t(., s, ,,) 
The two terms of Assoc try to capture different prop- 
erties of the SI{ expressed by the candidate class. Mu- 
tual information, \[(v;c/s), measures the strength of 
the statistical association between the given verb v 
and the candidate class c in the given syntactic po- 
sition s. If there is a real relationship, then hopefully 
l(v,c/s) >> 0. On the other hand, the conditional 
probability, P(e/v,s), favors those classes that have 
more occurrences of norms. 
2.4 Selecting the best classes 
The existence of noise in the training set introduces 
classes in tile candidate space that can't be considered 
as expressing SILs. A common technique used for ignor- 
ing ,as far ms possible this noise is to consider only those 
events that have a higher mnnber of occurrences than 
a certain threshold. Ilowever, some erroneous classes 
may persist because they exceed the threshold. How- 
ever, if candidate classes were ordered by the signif- 
icance of their Assoc with the verb, it is likely that 
less appropriate classes (introduced by noise) would 
be ranked in the last positions of the candidate llst. 
'\]'he algorithm to learn SIts is based in a search 
through all the ckmses with more instances in the train- 
ing set than the given threshold. In different iterations 
over ~hese candidate classes, two operations are per- 
fol:med: first, the class, c, having the best Assoc (best 
class), is extracted for tile final result; and second, the 
remaining candidate classes are filtered from classes be- 
ing hyper/hyponyms to the best class. This last step is 
made becanse the definitive classes must be mutually 
disjoint. The iterations are repeated until the candi- 
date space has been run otlt. 
771 
Table 1: SRs acquired for the subject of seek 
Acquired SR -~soc #n #s 
<cognition> Senses I -0.04 
< activity > Senses I ~0.01 
< status > Senses 0.087 
<social_control> Senses 0.111 
< administrative_district > Senses 0.14 
<city> Senses 0.15 
<radical> Senses 0.16 
< person, individual > Ok 0.23 
< legal_action > Ok 0.28 
< gro~p > ~}Abs. 0.35 
< suit > Senses 0.40 
< suit_of_clothes > Senses 0.41 
<suit,suing> Senses 0.41 
5 1 
6 1 
5 0 
6 0 
36 0 
36 0 
5 0 
61 38 
7 6 
64 46 
7 0 
7 0 
7 0 
Examples of nouns in Treebank 
concern, leadership, provision, science 
administration, leadership, provision 
government, leadership 
administration, government 
proper_name 
proper_name 
group 
advocate, buyer, carrier, client, company, ... 
suit 
administration, agency, bank, ..., group, ... 
suit 
suit 
suit 
\[Res92\] performed a similar learning process, but 
while he was only looking for the preferred class of ob- 
ject nouns, we are interested in all the possible closes 
(SRs). He performed a best-first search on the can- 
didate space. Itowever, if tile function to maximize 
doesn't have a monotone behavior (as it is the c~e of 
Assoc) the best-first search doesn't guarantee global 
optimals, but only local ones. This fact made us to 
decide for a global search, specially because the candi- 
date space is not so big. 
3 EXPERIMENTAL I{ESU LTS 
In order to experiment the methodology presented, we 
implemented a system in a Unix machine. The cor- 
pus used for extracting co-occurrence triples is a frag- 
ment of parsed material from the Penn Treebank Cor- 
pus (about 880,000 words and 35,000 sentences), con- 
sisting of articles of the Wall Street Journal, that has 
been tagged and parsed. We used Wordnet ~ the verb 
and noun lexicons for the lemmatizer, and also as the 
semantic taxonomy for clustering nouns in semantic 
classes. In this section we evaluate the performance of 
the methodology implemented: (1) looking at the per- 
formance of the techniques used for extracting triples, 
(2) considering the coverage of the WordNet taxonomy 
regarding the noun senses appearing in Treebank, and 
(3) analyzing the performance of the learning process. 
Tile total number of co-occurrence triples extracted 
amounts to 190,766. Many of these triples (68,800, 
36.1%) were discarded before tile lemmatizing pro~ 
tess because the surface NP head wasn't a noun. 
The remaining 121,966 triples were processed through 
the lemmatizer. 113,583 (93.1%) could be correctly 
mapped into their corresponding lemma \[\)rm. 
in addition, we analyzed manually the results ob- 
tained for a subset of tile extracted triples, looking 
at the sentences in the corpus where they occurred. 
The subset contains 2,658 examples of four average 
common verbs in the Treebank: rise, report, seek and 
present (from now on, tile testing sample). On the one 
hand, 235 (8.8%) of these triples were considered to 
be extracted erroneously because of the parser, and 51 
(1.9%) because of the lemmatizer. Summarizing, 2,372 
(89.2%) of the triples in the testing set were considered 
to be correctly extracted and lemmatized. 
When analyzing the coverage of WordNet taxono- 
my a we considered two different ratios. On the one 
hand, how many of the noun occurrences have one or 
more senses included in the taxonomy: 113,583 of the 
117,215 extracted triples (96.9%). On the other hand, 
how many of the noun occurrences in the testing sam- 
ple have the correct sense introduced in the taxonomy: 
2,615 of the 2372 well-extracted triples (8.7%). These 
figures give a positive evaluation of the coverage of 
WordNet. 
In order to evaluate the performance of the learn- 
ing process we inspected manually the SRs acquired 
on the testing-sample, a.ssessing if they corresponded 
to the actual Sits imposed. A first way of evaluation 
is by means of meazuring precision and recall ratios in 
the testing sample. In our e~e, we define precision as 
the proportion of triples appearing in syntactic posi- 
tions with acquired SRs, which effectively fififill one of 
those SRs. Precision amounts to 79.2%. The remain- 
ing 20.8% triples didn't belong to any of the classes 
induced for their syntactic positions. Some of them 
because they didn't have the correct sense included in 
the WordNet taxonomy, and others because tile cor- 
rect class had not been induced because there wasn't 
3The information of proper nouns in WordNet is poor. For 
this reason we assign four predel\]ned classes to them: 
< person, individual >, < organization :>, 
< adm{'t~iatrgtive_di.strict 2> etll(I <: city :>. 
772 
enough evidence. On the other hand, we dellne re- 
call as the proportion of triples which fnlfill one of tile 
SRs acquired for their corresponding syntactic posi- 
tions. Recall anrounts to 75.7%. 
A second way of evaluating the performance of t, he 
abstraction process is to manually diagnose the reasons 
that have made the system to deduce the SRs obtained. 
Table 1 shows the SILs corresponding to the subject 
position of the verb seek. Type indicates the diagnostic 
about the class appropriateness. Assoc, the value of 
the association score. "# n', tile number of nouns 
appearing in the corpus that are contained in the clmss. 
Finally, "~ s" "indicates the number of actual noun 
senses used in the corpus which are coutained in the 
class. In this table we can see some examples of the 
five types of manual diagnostic: 
Ok The acquired SR. is correct according to the noun 
senses contained in the corpus. 
~Abs The best level for stating the SI{ is not the one 
induced, but a lower one. It happens because er- 
roneous senses, irletollyInies, ...j accumulate evi- 
dence for the higher class. 
gAbs Some of the Slks could be best gathered in a 
unique class. We didn't lind any such case. 
Senses The class has cropped up because it accu- 
mulates enough evidence, provide.d by erroneous 
senses. 
Noise The class accumulates enough evidence pro- 
vided by erroneously extracted triples. 
Table 2 shows the incidence of the diagnostic types 
in the testing sample. Each row shows: the type of 
diagnostic, the numher and l)ercerttage of classes that 
accomplish it, and the nmnl)er and percentage of noun 
occurrences contained by these classes in tile testing 
sample 4. Aualyzing the results obtained from the 
testing sample (some of which are shown in tables 1 
and 2) we draw some positive (a, e) and some negative 
conclusions (b, c, d and /): 
a. Almost one correct semantic class tbr each syntac. 
tic position in the sample is acquired. The tech- 
nique aehicves a good coverage, even with few co-- 
occurrence triples. 
b. Although many of the classes acquired result Dora 
tile accumulation of incorrect senses (73.3%), it; 
seems that their size. tends to be smaller than 
cl~usses in other categories, an they only contain 
a 51,4% of tim senses . 
'lthls total doesn't equal the numher of triples in the testing 
sample because tile same 1t(2,1|11 iilgty belong to IllOFQ thali olll} 
class in the fin;d SIls 
Table 2: Summary of the Sits acquired 
Diagnostic # Clas~'es ~---~---~-n~----~-- 
Ok 45 
l}Abs 7 
liAbs 0 
Senses 176 
Noise 12 
Term 240 
2.9 362 I 6.8 
0.0 0 I 0.0 
73.3 2,7401 51.4 
5.0 130 I 2.4\[ 
 00. 0 
e. There doesn't seem to be a clear co-relation between 
Asset and the manual diagnostic. Specifically, the 
classes considered to be correct sometimes aren't 
ranked in t;he higher positions of tile Asset (e.g., 
Table l). 
(t. Tim over-generalization seems to be produced be- 
cause of little difference in the nouns included in 
the rival closes. Nevertheless this situation is 
rare. 
e. The impact of noise provided by erroneous e×trac- 
tion of cc~-occurrence triples, iu the acquisition of 
wrong semantic classes, seems to be very modero 
ate. 
(. Since diflhrcnt verb senses occur in the corpus, the 
SI{~ acquired appear mixed. 
4 \]?'U f\[TH EK V~r 0 Pd( 
Although perfornmnce of thc technique presented is 
pretty good, some of the detected problems could pose 
sibly be solved. Specifically, there are various ways to 
explore ill order to re.dace tile problems stated in points 
b and c above: 
1. 
2. 
To measure the Assoe by means of Mutual lntbr- 
marion between the pair v-s and c. In this way, 
tim syntactic position also wouhl provide iutbrma- 
lion (statistical evidence) for measuring the most 
appropriate classes. 
To modify tim Asset in such a way that it was 
based in a likelihood ratio test \[Dun93\]. It seems 
that this kind of tests have a better performance 
than nmtual inl'ormation when the counts are 
sma.ll, ~m it is the case. 
3. To estimate the probabilities of classes, not di+ 
rectly from the frequencies of their liouu men> 
bets, but correcting this evidence by the number 
of senses of those nouns, e.g 
#~ertJes ~,t ~C l'(~ls) ~ .)-'"co co.~,uv,. ~, ,m~,. +,-,-.v,J---"--'~ 
773 
In this way, the estimated function would be 
a probability distribution, and more interesting, 
nouns would provide evidence on the occurrence of 
their hyperonyms, inversely proportional to their 
degree of ambiguity. 
4. To collect a bigger number of examples for each 
verbal complement, projecting the complements 
in the internal arguments, using diathesis sub- 
categorization rules. Hopefully, Assoc would have 
a better performance if it was estimated on a big- 
ger population. On the other hand, in this way 
it Would be possible to detect the SRs holding on 
internal arguments. 
In order to solve point d above, we have foreseen two 
possibilities: 
1. To take into consideration the statistical signifi- 
cance of the alternatives involved, before doing a 
generalization step, climbing upwards, 
2. To use the PPs that in the corpus are attached 
to other complements and not to the main verb 
as a source of "implicit negative examples", in 
such a way that they would constrain the over- 
generalization. 
Finally, It would be interesting to investigate the so- 
lution to point fl One possible way would be to disam- 
biguate the senses of the verbs appering in the corpus, 
using the SRs already acquired and gathering evidence 
of the patterns corresponding to each sense by means 
of a technique similar to that used by \[Yar92\]. There- 
fore, once disambiguated the verb senses it would be 
possible to split the set of SRs acquired. 
\[BPV92\] 
\[CGHH91\] 
\[cH9o\] 
\[Dun93\] 
References 
P~. Basili,.M.T. Pazienza, and P. Velardi. 
Computationallexicons: the neat examples 
and the odd exemplars. In Proc. of lhe 3rd 
ANLP, 1992. 
K.W. Church, W. Gale, P. Hanks, and 
D. Itindle. Using statistics in lexicai anal- 
ysis. In U. Zernik, editor, Lexicat Acqui- 
sition: Exploiting On-Line Resources to 
Build a Lexicon. Lawrence Erlbaum, 1991. 
K.W. Church and P. Hanks. Word as- 
sociation norms, mutual information and 
lexicography. Computational Linguistics, 16(1), 
1990. 
T. Dunning. Accurate methods for the 
statistics of surprise and coincidence. Com- 
putational Linguistics, 19(1), 1993. 
\[MBF+90\] 
\[MSM93\] 
\[1%es92\] 
\[WFB90\] 
\[Yar92\] 
G. Miller, R. Beckwith, C. Fellbaum, 
D. Gross, and K. Miller. Five papers on 
wordnet. Technical report, CSL, Princeton 
University, 1990. 
Mitchell P. Marcus, Beatrice Santorini, 
and Mary Ann Marcinkiewicz. Building 
a large annotated corpus of english: the 
Penn Treebank. Computational Linguis- 
tics, 19(2), 1993. 
P. Resnik. Wordnet and distributional 
analysis: A class-based approach to lexi- 
cal discovery. In Proc. of AAAI Workshop 
on Statistical Methods in NLP, 1992. 
G. Whittemore, K. Ferrara, and II. Brun- 
net. Empirical study of predictive pow- 
ers of simple attachment schemes for post- 
modifier prepositional phrases. In Proc. of 
the 28th ACL, 1990. 
David Yarowsky. Word-sense disambigua- 
tion using statistical models of Roger's cat- 
egories trained on large corpora. In Pro- 
ceedings of COL\[NG-92, Nantes, France, 
1992. 
774 
