I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
Disambiguating Verbs 
with the WordNet Category of the Direct Object 
Eric V. Siegel 
Department of Computer Science 
Columbia University 
New York, NY 10027 
evs@cs, columbia, edu 
Abstract 
In this paper, I demonstrate that verbs can be dis- 
ambiguated according to aspect by rules that exam° 
hue the WordNet category of the direct object. First, 
when evaluated over a corpus of medical reports, I 
show that WordNet categories correlate with aspec- 
tual class. Then, I develop a rule for distinguishing 
between stative and event occurrences of have by the 
WordNet category of the direct object. This rule, 
which is motivated by both linguistic and statisti- 
cal analysis, is evaluated over an unrestricted set of 
nouns. I also show that WordNet categories improve 
a system that performs aspectual classification with 
linguistically-based numerical indicators. 
1 Introduction 
The verb have is semantically ambiguous. It can de- 
note a possessive relationship, as in, I had a car, or 
endow a quality, as in, I had anxiety. Further, have 
can describe an act of creation, as in, I had a baby, 
or an undertaking, as in, I had lunch. Broadly, all 
uses of have either denote a state, i.e., a situation 
that is not dynamic, or an event, i.e., a dynamic oc- 
currence that entails change or activity. This seman- 
tic distinction, stativity, is fundamental to many do- 
mainR, e.g., distinguishing symptoms and diagnoses 
from procedures in the medical domain. 
Stativity is the first distinction for the semantic 
hierarchy of verb phrases known as aspect. This hi- 
erarchy is linguistically established to enable reason- 
ing about time, i.e., temporal reasoning. Aspectual 
classification further distinguishes events according 
to completedness (i.e., telicity), which determines 
whether an event reaches a culmination point in time 
at which a new state is introduced. For example, I 
made a fire is culminated, whereas, I gazed at the 
sunset is non-culminated. 
Aspectual classification is necessary for interpret- 
ing temporal modifiers and assessing temporal en- 
tailments (Moens and Steedman, 1988; Dorr, 1992; 
Klavans, 1994), and is therefore a necessary com- 
ponent for applications that perform certain lan- 
guage interpretation, summarization, information 
retrieval, and machine translation tasks. Aspectual 
classification is a diflqcult problem because many 
verbs, like have, are aspectually ambiguous. 
In this paper, I demonstrate that verbs can be 
disambiguated according to aspect by the semantic 
category of the direct object. To this end, WordNet, 
the largest publicly available on-line lexical database 
(Miller et al., 1993), is used to provide semantic cat- 
egories for direct objects. When applied over a cor- 
pus of medical reports, I show that WordNet cate- 
gories correlate with aspectual class. Furthermore, 
I develop a rule for aspectual classification by the 
WordNet category of the direct object. This rule 
is specialized for the verb have, which presents a 
more prevalent disambiguation problem in medical 
reports than any other verb. The design of this rule 
is guided by both linguistic and statistical analy- 
sis. The rule is evaluated over an unrestricted set 
of nouns. WordNet categories are also shown to im- 
prove a system that performs aspectual classification 
with linguistically-based numerical indicators. 
The following section further discusses the seman- 
tic entailments of aspect and Section 3 discusses the 
problem of aspectual ambiguity. Section 4 describes 
the corpus used for this study, and Section 5 de- 
scribes our approach to disambiguating have. Sec- 
tion 6 then evaluates this approach and Section 7 de- 
scribes the use of WordNet for linguistic indicators. 
Finally, Section 8 provides conclusions and describes 
future work. 
2 Aspect in Natural Language 
Aspectual classification is a key component of mod- 
els that assess temporal constraints between clauses 
(Moens and Steedman, 1988; Hwang and Schubert, 
1991; Dorr, 1992; Hitzeman et al., 1994). For ex- 
ample, stativity must be identified to detect tempo- 
ral constraints between clauses connected with when. 
For example, in interpreting (I), 
(I) She had good strength when objectively tested. 
the following temporal relationship can hold between 
the have-state and the tear, event: 
have-strength 
i 
However, in interpreting (2), 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
Table I: Four aspectual markers and their linguistic 
constraints on aspectual class. 
If a clause can occur:, then it is: 
with a temporal adverb event 
(e.g., then) 
in prvyrwsive - event 
with a duration in-PP c-lminated 
(e.g., in an hour) event 
in the perfect tense c, lm. event 
or state 
(2) She had a se/zure when. objectively tested. 
the temporal relationship is between two events, and 
can be different: 
have-seizure 
I I 
Certain temporal adjuncts and tenses are con- 
strained by and contribute to the aspectual class 
of a clause (Vendler, 1967; Dowry, 1979). Tables 1 
lists four e.x_ample linguistic constraints. Each entry 
in this table describes an aspectual marker and the 
constraints on the aspectual category of any clause 
that appears with that marker. For example, if a 
clause appears in the progressive tense, it must be 
an event, e.g., 
(3) He is prospering. (event), 
which contrasts with, 
(4) *You are resembling your mother. (state). 
As a second example, an event must be culminated 
to appear in the perfect tense, for example, 
(5) She had made an attempt. (culminated), 
which contrasts with, 
(6) *He had stared at me. (non-c-lminated) 
Such constraints linguistically validate the aspec- 
tual hierarchy of semantic classes, provide seman- 
tic constraints for natural language generation and 
understanding, and provide guidelines for aspectual 
corpus analysis. 
3 Aspectually Ambiguous Verbs 
While some verbs appear to connote only one as- 
pectual class regardless of context, e.g., stare (non- 
c-lminated event), many verbs are aspectually am. 
biguous. For example, shaw denotes a state in, H/$ 
lumbar puncture showed evidence of white cells, but 
denotes an event in, He showed me the photographs. 
This ambiguity presents a di~culty for automati- 
cally classifying a verb because the aspectual class of 
a clause is a function of several clausal constituents 
in addition to the main verb (Dowry, 1979; Moens 
and Steedman, 1988; Pustejovsky, 1991). However, 
previous work that numerically evaluates aspectual 
classification has looked at verbs in isolation (Kla- 
vans and Chodorow, 1992; Siegel, 1997). 
10 
The verb have is particularly problematic. In the 
medical domain, have occurs as the main verb of 
clauses frequently (8% of clauses) and is aspectu- 
ally ambiguous, occurring 69.9% of the time as a 
state, and 30.1% of the time as an event. Most other 
ambiguous verbs are more highly dominated by one 
sense in this domain (Siegel, 1998). 
In this section, I examine factors contributing to 
aspectual ambiguity. First, I exam the interaction 
between a verb and its arguments in determining as- 
pectual class. The semantic category of open class 
words plays a large role in this process. And sec- 
ond, I describe a semantic hierarchy of statively am- 
biguous verb. This hierarchy groups together verbs 
that tend to interact with their arguments in similar 
ways. 
3.1 How Clausal Constituents Contribute 
to Aspectual Class 
The presence, syntactic categories, lexical heads, 
and plurality of a verb's arguments influence as- 
pectual class. This is illustrated in Table 2, which 
shows example clausal features that influence aspec- 
tual class. The effect of each feature is illustrated 
by showing two similar sentences with distinct as- 
pectual classes. 
The number of ways in which clausal constituents 
interactively influence aspect is ,mknown. However, 
syntax alone is not sufficient, and the lexical head 
of multiple constituents (e.g., the verb phrase and 
the direct object) are often factors. Moreover, the 
semantic category of these features can also play 
a role. For example, Sue played the piano is non- 
c,lminated, while Sue played the sonata signifies a 
c-lminated event (this example comes from Moens 
and Steedman (1988)). 
3.2 Classes of Ambiguous Verbs 
Placing aspectually ambiguous verbs into semantic 
categories will help predict how these verbs com- 
bine with their arguments to determine aspectual 
class. This is because many verbs with related mean- 
ings combine with their arguments in similar ways. 
In general, there is a correlation between a verb's 
subcategorization frame and semantic class (Levin, 
1993), and this applies to aspect in particular. 
For example, look and weigh can each appear as 
events, e.g., 
I looked at the baby. (event) 
I we/ghed the baby. (event) 
and can also appear as states, as in, 
The baby ~ heavy. (state) 
The baby weighed a lot. (state) 
Is this illustrates, these two verbs have similar sub- 
categorization frames that determine their aspectual 
class. There is also a relationship between their 
meanings, since each describes a type of perception 
or measurement. 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
Table 2: Example clausal features and how they can influence aspectual class. "P" means process (i.e., 
non-culminated event), "C" means culminated event, and "S" means state. 
Feature: 
"Predicate adj 
' Particle 
D~r obj cat 
~, Dir obj head 
Dir obj det 
, !nd obj det 
.Ind obj head 
Prep obj head 
Prep obj det 
Tense 
Example: class: 
John drove the car. P 
John drove the car. P 
John saw Sue. P 
Judith p/ayed the piano. P 
John ate fries. P 
Kathy sho~ed people her car. P 
Kathy showed people her car. P 
Judith looked around the store. P 
Kathy shot at deer. P 
Sal said that it helps. C 
Contrasting Example: 
John drove the car ragged. 
John drove the car up. 
John saw that Sue was happy. 
Judith p/ayed the sonata. 
John ate the fries. 
Kathy shorted the people her car. 
Kathy showed Sal her car. 
Judith looked around the corner. 
Kathy shot at the deer. 
Sal says that it helps. 
class: 
C 
C 
C 
C 
C 
C 
C 
C 
C 
S 
Group: 
coemunication 
cognition 
perception 
psy .¢h-movuant 
location 
metaphorical 
Table 3: Groups of verbs that axe statively ambiguous. 
Example verbs: 
admit, confirm, indicate, say 
judge, remember, think, wish 
feel, see, smell, weigh 
astonish, dismay, please, surlmdse 
hold, lie, sit, stand 
work, run 
continue , remain 
Event sentence: 
I said, #Hello." 
I thought about them. 
I felt the tablee/oth. 
You surprised me. 
I lay on the bed. 
I worked hard. 
\[ continued to talk about it. 
\[ State sentence: 
\[ say it is correct. 
I think they are nzce. 
I felt terrible. 
That suprises me. 
The book lies on the bed. 
The machine works. 
I continued to feel good. 
Table 3 shows the top level of a hierarchy of star 
tively ambiguous verbs. Seven semantic groups are 
shown, each with a set of example verbs, and two 
sentences illustrating contrasting uses of an example 
verb fxom that group. Each verb in the first group, 
communication, can appear as either an event or 
state. Intuitively, this is because each verb can con- 
vey a communicative act, e.g., 
She s.howed me the photos. (event) 
or, alternatively, a non-dynamic situation, e.g., 
The zrays show no sign ol ~rth. (state) 
Verbs in the second group in Table 3, cognitive, 
can convey a mental event, e.g., 
When he mentioned bananas, she remembered Ed- 
ward. (event) 
or, alternatively, a mental state, e.g., 
I'U ahvays remember Disney WorlcL (state) 
The groups perception and psych-movement are 
subgroups of cognition. The perception and 
communication groups have previously been ides- 
tiffed with respect to aspect in particular (Vendler, 
1967; Dowry, 1979), and those and psych-movement 
for general purposes beyond aspectual ambiguity 
(Levin, 1993). The fifth group, locative, has previ- 
ously been identified as "lay-verbs. ~ (Dowty, 1979) 
The group metaphorical in Table 3 contains event 
verbs with idiomatic uses that are stative. These id- 
iomatic uses correspond to a metaphorical interpre- 
tation of the event reading (Alexander D. Chaifee, 
personal communication). For example, 
I ra_.nn down the street. (event) 
It runs in the family. (state) 
Finally, cart/st verbs simply reflect the aspectual 
class of their clausal argument. 
4 Corpus: Medical Reports 
Our experiments are performed across a corpus of 
3,224 medical discharge summaries comprised of 
1,159,891 words. A medical discharge s,,mmary de- 
scribes the symptoms, history, diagnosis, treatment 
and outcome of a patient's visit to the hospital. As- 
pectual classification is necessary for several medical 
report processing tasks, since these reports describe 
events and states that progress over time (Friedman 
et al., 1995). 
These reports were parsed with the EngLish 
Slot Grammar (McCord, 1990), resulting in 97,973 
clauses that were parsed fully with no self-diagnostic 
errors (error messages were produced on some of 
this corpus' complex sentences). Parsing is needed 
to identify the main verb and direct object of each 
clause, as well as the presence of aspectual mark- 
ers for related statistical work, described below in 
Section 7. 
Be and have are the two most popular verbs, cov- 
ering 31.9% of the clauses in this corpus. Clauses 
with be as their main verb, composing 23.9% of the 
corpus, always denote a state. Clauses with have as 
their main verb, composing 8.0% of the corpus, are 
statively ambiguous. In this domain, most clauses 
with main verbs other than be and have can be aspec- 
tually classified by the the main verb only, e.g., by 
11 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
using numerical linguistic indicators (Siegel, 1998) 
In order to produce supertrised data with which 
to develop and evaluate our approach, a batch of 
206 have-clauses f~om the parsed corpus were man- 
ually marked according to stativity. As a linguistic 
test for marking, each clause was tested for read- 
ability with, What happened was... In a separate 
study, a comparison between two human markers us- 
ing this test to classify clauses over all verbs showed 
an agreement of approximately 91% (Siegel, 1998). 
The marked clauses, divided equally into training 
and testing sets of 103 clauses each, were used to 
develop and evaluate our approach, respectively. 
5 Applying WordNet 
I have manually designed a rule for classifying have- 
clauses according to stativity by the WordNet cat- 
egory of the direct object. To design this rule, the 
following were observed: 
• Distributions of objects of have over the corpus. 
• Linguistic intuition regarding WordNet cate- 
gories and aspectual class. 
• Correlations between the WordNet category of 
the direct object and stativity over the super- 
vised training data. 
To accumulate this information, WordNet was 
queried for each direct object of the parsed corpus. 
In particular, each noun was placed into one of the 
25 categories at the top of WordNet's semantic hi- 
erarchy, listed in Table 4. Many nouns have mul- 
tiple entries corresponding to multiple senses. As 
an initial approach, we take the first WordNet cate- 
gory listed, i.e., the most f~equent sense. Pronouns 
such as him and it were assigned their own category, 
pronoun. 
As shown, in Table 5, the most frequent objects 
of have are primarily specific to the medical domain. 
This table shows the high level semantic category 
assigned by WordNet and the classification of have- 
clauses with each noun as a direct object. WordNet 
is able to handle this technical domain since 89.1% 
of have-clauses have direct objects that are widely- 
known medical terms and non-technical terms. 
The rule shown in Table 6 classifies have-clauses 
based on the semantic category of their direct ob- 
ject. In particular, clauses with direct objects that 
belong to the categories event, act, phenomenon, 
communication, possession and food are classified 
as events, and all others are classified as states. 
Linguistic insights guided the design of this rule. 
For example, if the direct object of have denotes an 
event, such as seizure, the clause describes an event. 
For this reason, it is clear why the WordNet cate- 
gories event, act, phenomenon and communication 
each indicate an event clause. Note that nominalized 
event verbs, e.g., resolution, are placed in these four 
categories by WordNet. The category possession 
WordNet class 
location 0 
event 2 
act 6 
artifact 5 
phenomenon 2 
entity 2 
attribute 3 
meuu:e 3 
N/A 5 
cognition II 
state 19 
tJ.u 9 
substance 5 
relation 3 
person 2 
communication I 
causalagent 1 
posseesion 1 
group I 
food I 
shape 0 
natural object 0 
fenlin K 0 
aJD4m=l 0 
plant 0 
mot ivat ion 0 
as state a~ event 
1 
5 
6 
3 
1 
1 
1 
1 
1 
1 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
0 
Table 4: Word_Net categories of direct objects of have 
in the supervised training data. 
direct object n WordNet cl~ 
history 624 time 
ep/sode 280 event 
pain 192 cognition 
/@er 123 cognition 
temperature 113 attribute 
~lev~ 109 state 
movement 106 act 
course 96 act 
<none> 91 <none> 
symptom 81 cognition 
complaint 73 state 
s~z~re 72 event 
nausea 67 cogn£tion 
CI£Mm 
of clause 
state 
event 
state 
state 
*state 
state 
*event 
*event 
*state 
*state 
*state 
event 
*state 
Table 5: Frequent objects of have, their WordNet 
category, and the aspectual class of have-clauses 
with the object. Asterisks (*) denote classifications 
that were intuitively derived, since these examples 
did not occur in the training cases. 
was selected since, as shown in Table 6, most occur- 
rences of possession as a direct object of have are 
instances of loss, e.g., The patient had blood loss de- 
scribes an event. The category food was selected to 
cover idioms such as The patient had lunch (event). 
Furthermore, this classification rule is quantita- 
tively supported over the supervised training data. 
12 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
If then 
object is a(n): class is: n 
act event 1,157 
event 655 
phenomenon 242 
co~unication 194 
possession 59 
food 17 
cognition state 1,146 
state 875 
N/A 860 
time 636 
artifact 415 
attribute 349 
entity 209 
measuze 205 
substance 182 
relation 116 
person 115 
group g4 
location 49 
feeling 48 
pronoun 39 
animal 12 
Prequent nouns 
movement (106) course (96) di~iculty (66) scan (61) admission (60) 
episode (280) se/zure (72) pulse (28) recurrence (25) on.set (24) 
pressure (52) z-ray (30) flatus (21) response (19) intake (15) 
sign (25) resolution (22) effusion (18) section (17) electrocardiogram (12) 
loss (27) amount (15) res/dua/(5) insurance (4) cut (3) 
b~ (5) caH~ (2) ~min (1) ~,g,r (1) sco~ (1) 
pain (192) l~er (123) ~jmptnm (81) nausea (67) t~t (54) 
a/ler~ (109) complaint (73) infection (56) disesse (56) problem (40) 
echocardiogram (51) hematocr/t (41) ultrasound (34) stenosis (29) 
hist~.~ (624) r~m (8) paa (3) g~t/on (1) 
catheter (20) stool (19) tube (17) output (16) PPD (15) 
temperature (113) shortne~8 (46) tenderne.ss (26) levd (22) sound (16) 
chest (20) head (13) abdomen (13) artery (12) shunt (7) 
count (41) inc~,~se (18) bout (15) lull (12) day (9) 
blood (29) thallium (15) sodium (11) urine (10) fluid (9) 
change (40) rate (32) f~nct/on (12) aspirate (5) relationship (3) 
ch//d (13) aide (13) son (8) patient (8) temp (6) 
culture (41) serieJ (7) meet/m 3 (6) progression (4) panel (4) 
a~ (8) po~t (7) le/e (6) s~te (4) lab (4) 
appetite (18) relief(7) chill (6) preference (3) feeling (3) 
which (18) th/a (8) her (4) them (3) it (3) 
dog (3) paceer (2) pet (1) fetus (1) emu (1) 
Table 6: Aspectual classification rule for have-clauses. Counts are over all have-clauses in the medical reports 
corpus, from which the supervised training and testing data were extracted. 
For each WordNet category, Table 4 shows the distri- 
bution of event and stative have-clauses with a direct 
object belonging to that category. As shown, each 
WordNet category llnimd to states with our rule oc- 
curs at least as frequently in stative clauses as they 
do in event clauses within the training set, with the 
exception of coltmication, possession and food. 
However, these categories occur only one time each 
in the training data, which is too sparse to counter 
linguistic intuition. 
6 Results 
There is a strong correlation between the Word- 
Net category of a direct object, and the aspec- 
tual class of have-clauses it appears in. When us- 
ing the classification rule established in the previ- 
ous subsection, the WordNet categories that appear 
more than five times in the supervised test data 
correctly predict the class of have-clauses with an 
average precision of 82.7°/o. Specifically, act and 
event predict event have-clauses 85.7% and 66.7% 
correctly, respectively, and states are predicted with 
a~'l:ifact (62.5% precision), cognition (88.2%), 
state (93.3%) and t~ne (100.0%). 
For evaluating the rule's overall performance, 
there is a baseline of 69.9% and a ceiling of 84.5% 
accuracy. The baseline is achieved simply by classi- 
fying each clause as a state, since this is the domi- 
nant class over the supervised test cases, t However, 
XSimilar baselines for comparison have been used for many classification problems (Duds and Hart, 1973), e.g., part-of- 
I I overalll States Events 
acc recall prec recall prec 
C 84.5% 93.1% 85.9% 64.5% 80.0% 
R 79.6% 84.7% 85.9% 67.7% 65.6% 
B 69.9% 100.0% 69.9% 0.0% 100.0% 
Table 7: Performance of a rule (R) that uses the 
WordNet category of the direct object to aspectually 
classify have-classes, versus ceiling (C) and baseline 
(B) approaches. 
this approach classifies all event clauses incorrectly, 
achieving an event rr~21 of 0.0%. The ceiling of 
84.5% is the maximum achievable by a rule such as 
ours since the first WordNet category of the direct 
object is not always enough to resolve aspectual am- 
biguity; the same category appears in both stative 
and event test cases. 
Overall classification performance using Word- 
Net categories is greatly improved over the baseline 
method. As shown in Table 7, an accuracy of 79.6% 
was achieved. A binomial test showed that this im- 
provement over the baseline is significant (p < .04). 
An event greater improvement over the baseline 
is illustrated by the increase in the number of event 
clauses correctly classified, i.e. event rr£all. As 
shown in Table 7, an event recall of 67.7% was 
achieved by the classification rule, as compared to 
speech tagging (Church, 1988; Alien, 1995). 
13 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
I 
the 0.0% event recall achieved by the baseline, while 
suffering no loss in overall accuracy. This differ- 
ence in recall is more dramatic than the accuracy 
improvement because of the dominance of stative 
clauses in the test set. A favorable tradeoff in re- 
call with no loss in accuracy presents an advantage 
for applications that weigh the identification of non- 
dominant instances more heavily (Cardie and Howe, 
1997). For example, it is advantageous for a medical 
system that identifies medical procedures to identify 
event clauses, since procedures are a type of event. 
There are several problematic cases that illustrate 
limitations to our approach. In particular, lexical 
ambiguity is mi.qleading for the task of classifying 
have-clauses. For example, The paticnt had Med/c~/d 
denotes a state, but WordNet categorizes Med/ca/d 
as an act. Similarly, PET, EMUand CATare cate- 
gorized as animal, as shown in Table 6. This would 
be solved by recognizing these as proper nouns or 
acronyms due to capitalization. However, other am- 
biguous objects are more difficult to address. For 
e~Ample, The patient had an enema describes an 
event, but WordNet lists enema as artifacl; be- 
fore act. As another example, The patient had a 
urine culture is an event, but WordNet's first sense 
of cu/tuw is group. Furthermore, the direct object of 
10.9% of have-clauses in the medical reports are un- 
known to WordNet ("N/A"). This includes medical 
terminology, e.g., anticont~ants and vitrectomy, as 
well as certain expressions parsed by the English Slot 
Grammar that require further post-processing, such 
as bettoeen 39 and 29. 
7 WordNet for Linguistic Indicators 
Aspectual classification is a large-scale, domain- 
dependent problem. Although a complete aspectual 
lexicon of verbs may suffice to classify many clauses 
by their main verb only, a verb's primary class is 
often domain-dependent. For example, while many 
dom~inR primarily use show as an event, its appear- 
ances in medical discharge snmmaxies primarily de- 
note states. Therefore, it is necessary to produce a 
specialized lexicon for each domain. 
One statistical approach is to measure linguistic 
indicators over a corpus (Siegel, 1998). These in- 
dicators measure how frequently each verb appears 
with markers such as those in Table 1. For exam- 
ple, a verb that appears more frequently in the pro- 
gressive is more likely to describe an event than a 
state (Klavans and Chodorow, 1992). However, this 
approach attempts to classif T verbs independent of 
their context. 
Incorporating additional constituents of a clause 
could alleviate this problem. For example, indicators 
could be measured over verb-object pairs. However, 
since both the main verb and the head of the direct 
object are open-class categories, indicators would be 
sparsely measured (enjopturnips is rare). 
To alleviate sparsity, but retain information about 
\[ \[overall I Culm Non-Culm 
ace redall prec recall prec 
W 71.1% 81.5% 75.0% 53.1% 62.5% 
V 68.5% 86.2% 70.6% 38.1% 61.4% 
B 63.3% 100.0% 63.3% 0.0% 100.0% 
Table 8: Comparison of indicators computed over 
the main verb (V), indicators over verb and object's 
WordNet category pairs (W), and a baseline (B). 
the direct object, we measured indicators over verb- 
object-category pairs, using WordNet to derive the 
semantic category of each object. I describe such 
experiments briefly here; Further details regarding 
these experiments is given by Siegel (1998). 
Fourteen such indicators were evaluated for distin- 
guishing clauses according to completednese over an 
unrestricted set of verbs and direct objects. A cor- 
pus of 75,289 parsed clauses from ten novels was used 
to measure indicator values. 307 training cases (196 
culminated) and 308 test cases (195 culminated) 
were manually annotated using linguistic tests. De- 
cision tree induction was performed over the training 
cases to combine the indicators. 
Indicators measured over the main verb and di- 
rect object category achieved a more favorable re- 
call tradeoff than those measured over the verb 
only, with comparable performance in accuracy. As 
shown in Table 8, indicators measured over the main 
verb and direct object category achieved a non- 
culminated recall of 53.1%, as compared to 38.1% 
achieved by the verb-only indicators. The baseline 
of 63.3% accuracy is achieved by simply classifying 
every clause as culminated. 
8 Conclusions and Future Work 
The semantic category of the direct object plays a 
major role in determining the aspectual class of a 
clause. To demonstrate this, a rule was developed 
that uses WordNet categories to classify have-clauses 
according to stativity. When evaluated over an unre- 
stricted set of nouns, this rule achieved an accuracy 
of 79.6%, compared to the baseline performance of 
69.9%. Moreover, a favorable tradeoff in recall was 
achieved, attaining 67.7% event recall, compared to 
the the baseline's 0.0%. More specifically, frequent 
WordNet categories were shown to predict aspectual 
class with an average precision of 82.7%. These re- 
sults are impressive, considering the unresolved se- 
mantic ambiguity of direct objects, and the technical 
terminology of the medical domain. 
WordNet categories also improved the classifica- 
tion performance of linguistic indicators for com- 
pletedness. Although more sparsely measured, the 
accuracy achieved by indicators measured over mul- 
tiple constituents is comparable to that of indicators 
measured over the verb only, with a favorable trade- 
14 
I 
I 
I 
I 
I 
I 
I 
I 
I 
off in recall. Therefore, the noise introduced by this 
more sparse measurement of indicators is more than 
compensated for by the ability to resolve aspectually 
ambiguous verbs. 
Furthermore, I have derived a semantic hierar- 
chy of statively ambiguous verbs in order to predict 
verbs' subcategorization frames. This in turn guides 
the disambiguation of such verbs. Future work will 
investigate whether rules such as that developed for 
have could apply over multiple verbs that share sub- 
categorization behavior. Additionally, it is possible 
that WordNet's categorization of verbs could auto- 
matically place verbs into these semantic groups. 
Finally, disambiguating the direct object accord- 
ing to WordNet categories, e.g., Resnik (1995), 
would improve the accuracy of using these categories 
to disambiguate verbs. 
Acknowledgements 
Kathieen IL McKeown was extremely helpful regard- 
ing the formulation of our work and Judith Klavans 
regarding linguistic techniques. Alexander D. Char- 
fee, Vasileios Hatzivassiloglou, Dragomir Radev and 
Dekai Wu provided many helpful insights regard- 
ing the evaluation and presentation of our results. 
James Shaw first brought to my attention that have 
is statively ambiguous, and, along with Eleazar Es- 
kin and Regina Barzilay, provided useful feedback 
on an earlier draft of this paper. 
This research is suppoi'ted in part by the 
Columbia University Center for Advanced Technol- 
ogy in High Performance Computing and Commu- 
nications in Healthcare (funded by the New York 
State Science and Technology Foundation), the Of- 
rice of Naval Research under contract N00014-95-1- 
0745 and by the National Science Foundation under 
contract GER-90-24069. 

References 
J. Allen. 1995. Natural Language Understanding. 
Benjamin/Cummlngs, Redwood City, CA. 
C. Cardie and N. Howe. 1997. Improving mi- 
nority class prediction using case-specific feature 
weights. In D. Fisher, editor, Proceedings o/the 
Fourteenth International Conference on Machine 
Learning. Morgan Kaufmann. 
K. Church. 1988. A stochastic parts program and 
noun phrase parser for unrestricted text. In Pro- 
ceedim2s of the '2nd Conference for Applied Natural 
Language Prvcessing, pages 136--143. 
B.J. Doff. 1992. A two-level knowledge representa- 
tion for machine translation: lexical semantics and 
tense/aspect. In James Pustejovsky and Sabine 
Bergler, editors, ~ Semantics and Knowledge 
Representation. Springer Verlag, Berlin. 
D.R. Dowty. 1979. Word Meaning and Montague 
Grammar. D. Reidel, Dordrecht, W. Germany. 
R. O. Duda and P.E. Hart. 1973. Pattern Classifi- 
cation and Scene Analysis. Wiley, New York. 
C. b'~'iedman, G. Hripcsak, W. DuMouchel, S.B. 
Johann, and P.D. Clayton. 1995. Natural lan- 
guage processing in an operational clinical infor- 
mation system. Natural Language Engineering, 2(1). 
J. Hitzeman, M. Moens, and C. Grover. 1994. Al- 
gorithrrm for analysing the temporal structure of 
discourse. Technical report, University of Edin- 
burgh. 
C.H. Hwang and L.K. Schubert. 1991. Interpreting 
temporal adverbials. Technical report, University 
of Rochester. 
J.L. Klavans and M. Chodorow. 1992. Degrees of 
stativity: the lexical representation of verb aspect. 
In Proceedings of the 1Jth International Confer- 
enee on Computation Linguistics. 
J.L. Klavans. 1994. Linguistic tests over large cor- 
pora: aspectual classes in the lexicon. Technical 
report, Columbia University Dept. of Computer 
Science. unpublished manuscript. 
B. Levin. 1993. English Verb CIasses and Alterna- 
tions. University of Chicago Press, Chicago, 11,. 
M.C. McCord. 1990. SLOT GRAMMAR. In 
It. Studer, editor, International Symposium on 
Natural Language and Logic. Springer Verlag. 
G.A. Miller, R. Beckwith, C. Felbaum, D. Gross, and 
K. Miller. 1993. Introduction to wordnet: An on- 
line lexical database. Technical report. 
M. Moens and M. Steedman. 1988. Temporal ontol- 
ogy and temporal reference. Computational Lin- 
guist/es, 14(2). 
J. Pustejovsky. 1991. The syntax of event structure. 
Cognition, 41(103):47-92. 
P. Resnik. 1995. Disambiguating noun groupings 
with respect to WorclNet senses. In Third Work- 
shop on Very Large Corpora, June. 
E.V. Siegel and K.R. McKcown. 1996. Gathering 
statistics to aspectually classify sentences with a 
genetic algorithm. In K. Oflazer and H. Somers, 
editors, Proceedings of the Second Inter'national 
Conference on New Methods in Language Process- 
ing, Ankara, Turkey, Sept. Bilkent University. 
E.V. Siegel. 1997. Learning methods for combining 
linguistic indicators to classify verbs. In Prvceed- 
ings of the Second Conference on Empirical Meth- 
ads in Natural Language Processing, Providence, 
RI, August. 
E.V. Siegel. 1998. Linguistic Indicators for Lan- 
guage Understanding: Using machine learning 
methods to combine corpus-based indicators for 
aspectual classification of clauses. Ph.D. thesis, 
Columbia University. 
Z. Vendler. 1967. Verbs and times. In Linguistics in 
Philosophy. Cornell University Press, Ithaca, NY. 
