<?xml version="1.0" standalone="yes"?>
<Paper uid="E91-1039">
  <Title>AUTOMATIC SEMANTIC CLASSIFICATION OF VERBS FROM THEIR SYNTACTIC CONTEXTS: AN IMPLEMENTED CLASSIFIER FOR STATIVITY</Title>
  <Section position="1" start_page="0" end_page="0" type="metho">
    <SectionTitle>
AUTOMATIC SEMANTIC CLASSIFICATION OF VERBS FROM THEIR
SYNTACTIC CONTEXTS: AN IMPLEMENTED CLASSIFIER FOR STATIVITY
</SectionTitle>
    <Paragraph position="0"/>
  </Section>
  <Section position="2" start_page="0" end_page="0" type="metho">
    <SectionTitle>
MIT AI Lab
545 Technology Square
</SectionTitle>
    <Paragraph position="0"/>
  </Section>
  <Section position="3" start_page="0" end_page="0" type="metho">
    <SectionTitle>
Abstract
</SectionTitle>
    <Paragraph position="0"> This paper discusses an implemented program that automatically classifies verbs into those that ~ describe only states of the world, such as to know, and those that describe events, such as to look. It works by exploiting the con, straint between the syntactic environments in which a verb can occur and its meaning. The only input is on-line text. This demonstrates an important new technique for the automatic generation of lexical databases.</Paragraph>
  </Section>
  <Section position="4" start_page="0" end_page="0" type="metho">
    <SectionTitle>
1 Introduction
</SectionTitle>
    <Paragraph position="0"> Young children and natural language processing programs face a common problem: everyone else knows a lot more about words. Children, it is hypothesized, catch up by observing the linguistic and non-linguistic contexts in which words are used. This paper focuses on the value and accessibility of the linguistic context. It argues that linguistic context by itself can provide useful cues about verb meaning to an artificial learner. This is demonstrated by a program that exploits two particular cues from the linguistic context to classify verbs automatically into those whose sole sense is one describing a state, and those that have a sense describing an event. 1 The approach described here accounts for a certain degree of noise in the input due both to mis-apprehension of input sentences and to their occasional real.formation. This work shows that the two cues are available and are reliable given the statistical methods applied.</Paragraph>
    <Paragraph position="1"> Language users, whether natural or artificial, need detailed semantic and syntactic classifications of words. Ultimately, any artificial language IThe input sentences are those compiled in the Lancaster/Oslo/Bergen (LOB) Corpus, a balanced corpus of one million words of British English. The LOB consists primarily of edited prose.</Paragraph>
    <Paragraph position="2"> user must be able to add new words to its lexicon, if only to accommodate the many neologisms it will encounter. And our lexicographic needs grow with our understanding of language. A number of current approaches to satisfying the lexical requirements for artificial devices do not involve unsupervised learning from examples. Boguraev and Briscoe (1987)discusses interpreting the information published in on-line dictionaries, while Zernik a/~d Dyer (1987) discuss tutored learning in a con.</Paragraph>
    <Paragraph position="3"> trolled environment. But any method that requires explicit human intervention -- be it that of lexicographers, knowledge engineers, or &amp;quot;tutors&amp;quot; -- will lag behind both the growth of vocabulary and the growth of linguistics, and even with the lag their maintenance will be expensive. By contrast, dictionaries constructed by automated learners from real sentences will not lag behind vocabulary growth; examples of current language use are free and nearly infinite. These observations have led ~everal researchers, including Hindle (1990) and Smadja and McKeown (1990), to begin investigating automatic acquisition of semantics.</Paragraph>
    <Paragraph position="4"> Hindle and Smadja and McKeown rely purely on the ability of one particular word to statistically predict the occurrence of another in a particular position. In contrast, the approach described here is targeted at particular semantic classes that are revealed by specific linguistic constructions.</Paragraph>
  </Section>
  <Section position="5" start_page="0" end_page="0" type="metho">
    <SectionTitle>
2 The Questions
</SectionTitle>
    <Paragraph position="0"> This section discusses work on two linguistic cues that reveal the availability of non-stative senses for verbs. This work attempts to determine the difficulty of using the cues to classify verbs: into those describing states and those describing  events. To that end, it focuses on two questions: 1. Is it possible to reliably detect the two cues  using only a simple syntactic mechanism and minimal syntactic knowledge? How simple can the syntax be? (The less knowledge required to learn using a given technique, the</Paragraph>
    <Paragraph position="2"> more useful the technique will be.) 2. Assuming minimal syntactic power, how reliable are our two cues in real text, which is subject to performance limitations? Are there learning strategies under which their reliability is adequate? Section 2.1 describes syntactic constructions studied and demonstrates their relation to the stative semantic class. Sections 2.2 answers questions 1 in the affirmative. Section 2.4 answers question 2 in the affirmative, discusses the statistical method used for noise reduction, and demonstrates the program that learns the state-event distinction.</Paragraph>
    <Section position="1" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
2.1 l'teveaHng Constructions
</SectionTitle>
      <Paragraph position="0"> The differences between verbs describing states (statives) and those describing events (nonstatives) has been studied by linguists at least since Lakoff (1965). (For a more precise semantic characterization of stativC/s see Dowty, 1979.) Classic examples of stative verbs are know, believe, desire, and love. A number of syntactic tests have been proposed to distinguish between statives and non-statives (again see Dowry, 1979). For example, stative verbs are anomalous when used in the progressive aspect and when 'modified by rate adverbs such as quickly and slowly:  (1) a. * Jon is knowing calculus b. * Jon knows calculus quickly Perception verbs like see and hear share with st~rives a strong resistance to the progressive aspect, but not to rate adverbs: (2) a. * Jon is seeing the car b. OK Jon quickly saw the car Agentive verbs describing attempts to gain perceptions, like look and listen, do not share either property: (3) a. OK Jon is looking at a car b. OK Jon quickly looked at his watch  The classification program relies primarily on the progressive cue, but uses evidence from the rate adverb cue when it is available.</Paragraph>
    </Section>
    <Section position="2" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
2.2 Syntactic Requirements for Cue
Detection
</SectionTitle>
      <Paragraph position="0"> Consider first how much syntactic analysis is needed to detect the progressive and rate adverb constructions. Initially, suppose that the availability of a non-stative sense is aii intrinsic property of a verb 2 not affected by its syntactic context. To detect progressives one need only parse a trivial part of the auxiliary system, which is known to  that think that.., is stative while think about.., is not. be finite-state. Detecting the rate aclverb cue requires determining what the adverb modifies, and that can be trickier. For example, adverbs may appear after the direct object, (4a), and this must not be confused with the case where they appear after the subject of an embedded clause, (4b).</Paragraph>
      <Paragraph position="1"> (4) a. ,Ion fixed the robot quickly b. ,Ion knew his hostess rapidly lost intereat in such things  Using simple, finite-state machinery one would be forced to deal with (4b) by recognizing the position of the adverb as ambiguous and rejecting the example. Or one could deploy more sophist i cated syntax to try determining the boundaries of embedded sentences. But even the best syntactic parser will fail on truly ambiguous cases like the following:  (5) a. Jon fixed the robot that had spoken slowly b. Jon believed the robot that had spoken slowly  The data on rate adverbs were collected using the parsing approach, which required a substantial amount of machinery, but a finite-state approach might do almost as well. (See Brent and Berwick, 1991, for automatic iexical acquisition using simple finite-state parsing.)</Paragraph>
    </Section>
    <Section position="3" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
2.3 Data on Cues from the Corpus
</SectionTitle>
      <Paragraph position="0"> To test the power of the two proposed cues, the LOB corpus was automatically processed to determine what percentage of each verb's occurrences were in the progressive, and what percent. age were modified by rate adverbs. Sampling error was handled by calculating the probability distribution of the true percentage for each verb assuming that the sentences in the corpus were drawn at random from some infinitely large corpus. The overall frequency of the progressive construction was substantially higher than that of the rate adverb construction and so provided more significant data. Figure 1 shows a histogram constructed by summing these distributions of true frequency in the progressive over each of the 38 most common verbs in the corpus. 3 Figure 1 shows that, at least for these most common verbs, there are three and perhaps four distinct populations. In other words, these verbs do not vary continuously in their frequency of occurrence in the progressive, but rather show a marked tendency to cluster around certain values. As will be shown in the next section, the 3Histograms that include less frequent verbs have the same general character, but the second local maximum gets somewhst blurred by the many verbs whose true frequency in the progressive is poorly localized due to insufficient sample size.</Paragraph>
      <Paragraph position="2"> in the progressive over each of the 38 most common verbs in the corpus stative verbs fall in the first population, to the left of the slight discontinuity at 1.35% of occurrence in the progressive.</Paragraph>
    </Section>
    <Section position="4" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
2,4 The Classification Program
</SectionTitle>
      <Paragraph position="0"> 1 implemented a program that attempts to classify verbs into those with event senses, and those whose only meaning describes a state rather than an event. It does this by first detecting occurrences of the progressive and rate adverb constructions in the LOB corpus, and then computing confidence intervals on the true frequency of occurrence of each verb in an arbitrarily large corpus of the same composition. The program classifies the verbs according to bounds, which are for the moment supplied by the researcher, on the confidence intervals. For example, on the run shown in Figure 2, the program classifies verbs which occur at least .1% of the time either in the progressive or modified by a rate adverb, as having an event (non-stative) sense. The classifier acts on .1% bound only if the sample-size is large enough to guarantee the bound with 95% confidence. Accuracy in ascribing non-stative senses according with this technique is excellent -- no purely sta.</Paragraph>
      <Paragraph position="1"> tire verbs are ntis-classified as having non-stative senses. In fact, this result is not very sensitive to raising the minimum progressive frequency from .1% to as high as .6% or .7%, since most verbs with non-stative senses yield observed frequencies of at least two or three percent.</Paragraph>
      <Paragraph position="2"> Now consider the other side of the problem, classifying verbs as purely stative. Here the program takes verbs that fail the test for having a non-stative sense, and in addition whose true frequency in the progressive falls below a given upper bound with sufficient confidence. The rate-adverb construction is not used, except insofar as the verbs must fail the . 1% lower bound, because this construction turns out to be so rare that only a few of the most frequent verbs provide sufficiently tight bounds. The results for identifying pure sta- null 100 times in the LOB corpus tives are also quite good, although more sensitive to the precise bounds than were the results for identifying non-statives. If the upper bound on progressive frequency is set at 1.35%, as in Figure 2, then eleven verbs are identified as purely stative, of the 204 distinct verbs occurring at least 100 times each in the corpus. Two of these, hear and agree, have relatively rare non-stative senses, meaning to imagine one hears (&amp;quot;I am hearing a ringing in my ears&amp;quot;) and to communicate agreement (&amp;quot;Rachel was already agreeing when Jon interrupted her with yet another tirade&amp;quot;). If the upper bound on progressive frequency is tightened to 1.20% then hear and agree drop into the &amp;quot;indeterminate&amp;quot; category of verbs that pass neither test. So, too, do three pure statives, mean, require, and understand.</Paragraph>
      <Paragraph position="3"> It is worth noting the importance of using some sort of noise reduction technique, such as the confidence intervals used here. There are two sources of noise in the linguistic input. First speakers do utter anomalous sentences. For ex.</Paragraph>
      <Paragraph position="4"> ample, the stative verb mean occurred one time out of 450 in the progressive. The sentence, &amp;quot;It's a stroke, that was what he was meaning&amp;quot; is clearly anomalous. The second source of noise is failure of the learner to detect the cue accurately. The accuracy of our automatic cue detection detection is described in the following section.</Paragraph>
    </Section>
    <Section position="5" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
2.5 Accuracy of Cue Detection
</SectionTitle>
      <Paragraph position="0"> Section 2.2 discussed how much structure must be imposed on sentences if the progressive and rate-adverb constructions are to be detected. Section 2.3 showed that the progressive and rate-adverb constructions are indeed reliable cues for the availability of a non-stative sense. This section discusses tile accuracy with which these cues can be detected.</Paragraph>
      <Paragraph position="1"> It is not practical to check manually every verb occurrence that our program judged tO be progressive. Instead, I checked 300 such sentences  selected at random from among the most commonly occurring verbs. This check revealed only one sentence that did not truly describe a progressive event. That sentence is shown in (6a). (6) a. go: What that means in this case is go.</Paragraph>
      <Paragraph position="2"> ing back to the war years...</Paragraph>
      <Paragraph position="3"> b. see: The task was solely to see how speedily it could be met...</Paragraph>
      <Paragraph position="4"> c. compare: ...the purchasing power of the underdeveloped countries in the commonwealth will rise slowly compared with that of Europe.</Paragraph>
      <Paragraph position="5"> It is not clear how to automatically determine that (6a) does not describe an event of going in progress. Rate adverbs are infrequent enough that it was possible to verify manually all 281 cases the program found. In four of those cases the rate adverb actually modified a verb other than the one that the program chose. Three of these four cases had the structure of (6b), where a wh- relative is not recognized as signaling the beginning of a new clause. This reflects an oversight in the grammar that should be easily correctable. The one remaining case of a mis-attributed rate adverb, (6c), would again require some work, and perhaps substantial syntactic knowledge, to correct. The rate of false positives in cue detection, then can be estimated at about one serious hazard in 300 for both &amp;quot;t'~sts.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>