File Information
File: 05-lr/acl_arc_1_sum/cleansed_text/xml_by_section/metho/91/p91-1027_metho.xml
Size: 16,411 bytes
Last Modified: 2025-10-06 14:12:49
<?xml version="1.0" standalone="yes"?> <Paper uid="P91-1027"> <Title>AUTOMATIC ACQUISITION OF SUBCATEGORIZATION FRAMES FROM UNTAGGED TEXT</Title> <Section position="1" start_page="0" end_page="0" type="metho"> <SectionTitle> AUTOMATIC ACQUISITION OF SUBCATEGORIZATION FRAMES FROM UNTAGGED TEXT </SectionTitle> <Paragraph position="0"/> </Section> <Section position="2" start_page="0" end_page="0" type="metho"> <SectionTitle> MIT AI Lab 545 Technology Square </SectionTitle> <Paragraph position="0"/> </Section> <Section position="3" start_page="0" end_page="0" type="metho"> <SectionTitle> ABSTRACT </SectionTitle> <Paragraph position="0"> This paper describes an implemented program that takes a raw, untagged text corpus as its only input (no open-class dictionary) and generates a partial list of verbs occurring in the text and the subcategorization frames (SFs) in which they occur. Verbs are detected by a novel technique based on the Case Filter of Rouvret and Vergnaud (1980). The completeness of the output list increases monotonically with the total number of occurrences of each verb in the corpus. False positive rates are one to three percent of observations. Five SFs are currently detected and more are planned. Ultimately, I expect to provide a large SF dictionary to the NLP community and to train dictionaries for specific corpora.</Paragraph> </Section> <Section position="4" start_page="0" end_page="0" type="metho"> <SectionTitle> 1 INTRODUCTION </SectionTitle> <Paragraph position="0"> This paper describes an implemented program that takes an untagged text corpus and generates a partial list of verbs occurring in it and the sub-categorization frames (SFs) in which they occur.</Paragraph> <Paragraph position="1"> So far, it detects the five SFs shown in Table 1.</Paragraph> <Paragraph position="2"> The SF acquisition program has been tested on a corpus of 2.6 million words of the Wall Street Journal (kindly provided by the Penn Tree Bank project). On this corpus, it makes 5101 observations about 2258 orthographically distinct verbs. False positive rates vary from one to three percent of observations, depending on the SF.</Paragraph> <Section position="1" start_page="0" end_page="0" type="sub_section"> <SectionTitle> 1.1 WHY IT MATTERS </SectionTitle> <Paragraph position="0"> Accurate parsing requires knowing the sub-categorization frames of verbs, as shown by (1).</Paragraph> <Paragraph position="1"> (1) a. I expected \[nv the man who smoked NP\] to eat ice-cream h. I doubted \[NP the man who liked to eat ice-cream NP\] Current high-coverage parsers tend to use either custom, hand-generated lists of subcategorization frames (e.g., Hindle, 1983), or published, hand-generated lists like the Ozford Advanced Learner's Dictionary of Contemporary English, Hornby and Covey (1973) (e.g., DeMarcken, 1990). In either case, such lists are expensive to build and to maintain in the face of evolving usage. In addition, they tend not to include rare usages or specialized vocabularies like financial or military jargon. Further, they are often incomplete in arbitrary ways. For example, Webster's Ninth New Collegiate Dictionary lists the sense of strike meaning 'go occur to&quot;, as in &quot;it struck him that... &quot;, but it does not list that same sense of hit. (My program discovered both.)</Paragraph> </Section> </Section> <Section position="5" start_page="0" end_page="209" type="metho"> <SectionTitle> 1.2 WHY IT'S HARD </SectionTitle> <Paragraph position="0"> The initial priorities in this research were: . Generality (e.g., minimal assumptions about the text) . Accuracy in identifying SF occurrences * Simplicity of design and speed Efficient use of the available text was not a high priority, since it was felt that plenty of text was available even for an inefficient learner, assuming sufficient speed to make use of it. These priorities had a substantial influence on the approach taken. They are evaluated in retrospect in Section 4. The first step in finding a subcategorization frame is finding a verb. Because of widespread and productive noun/verb ambiguity, dictionaries are not much use -- they do not reliably exclude the possibility oflexical ambiguity. Even if they did, a program that could only learn SFs for unambiguous verbs would be of limited value. Statistical disambiguators make dictionaries more useful, but they have a fairly high error rate, and degrade in the presence of many unfamiliar words. Further, it is often difficult to understand where the error is coming from or how to correct it. So finding verbs poses a serious challenge for the design of an accurate, general-purpose algorithm for detecting SFs. In fact, finding main verbs is more difficult than it might seem. One problem is distinguishing participles from adjectives and nouns, as shown below.</Paragraph> <Paragraph position="1"> (2) a. John has \[~p rented furniture\] (comp.: John has often rented apartments) null b. John was smashed (drunk) last night (comp.: John was kissed last night) c. John's favorite activity is watching TV (comp.: John's favorite child is watching TV) In each case the main verb is have or be in a context where most parsers (and statistical disambiguators) would mistake it for an auxiliary and mistake the following word for a participial main verb.</Paragraph> <Paragraph position="2"> A second challenge to accuracy is determining which verb to associate a given complement with. Paradoxically, example (1) shows that in general it isn't possible to do this without already knowing the SF. One obvious strategy would be to wait for sentences where there is only one candidate verb; unfortunately, it is very difficult to know for certain how many verbs occur in a sentence. Finding some of the verbs in a text reliably is hard enough; finding all of them reliably is well beyond the scope of this work.</Paragraph> <Paragraph position="3"> Finally, any system applied to real input, no matter how carefully designed, will occasionally make errors in finding the verb and determining its subcategorizatiou frame. The more times a given verb appears in the corpus, the more likely it is that one of those occurrences will cause an erroneous judgment. For that reason any learning system that gets only positive examples and makes a permanent judgment on a single example will always degrade as the number of occurrences increases. In fact, making a judgment based on any fixed number of examples with any finite error rate will always lead to degradation with corpussize. A better approach is to require a fixed percentage of the total occurrences of any given verb to appear with a given SF before concluding that random error is not responsible for these observations. Unfortunately, determining the cutoff percentage requires human intervention and sampling error makes classification unstable for verbs with few occurrences in the input. The sampling error can be dealt with (Brent, 1991) but predetermined cutoff percentages stir require eye-bailing the data. Thus robust, unsupervised judgments in the face of error pose the third challenge to developing an accurate learning system.</Paragraph> </Section> <Section position="6" start_page="209" end_page="209" type="metho"> <SectionTitle> 1.3 HOW IT'S DONE </SectionTitle> <Paragraph position="0"> The architecture of the system, and that of this paper, directly reflects the three challenges described above. The system consists of three modules: 1. Verb detection: Finds some occurrences of verbs using the Case Filter (Rouvret and Vergnaud, 1980), a proposed rule of grammar. null 2. SF detection: Finds some occurrences of five subcategorization frames using a simple, finite-state grammar for a fragment of English. null 3. SF decision: Determines whether a verb is genuinely associated with a given SF, or whether instead its apparent occurrences in that SF are due to error. This is done using statistical models of the frequency distributions. null The following two sections describe and evaluate the verb detection module and the SF detection module, respectively; the decision module, which is still being refined, will be described in a subsequent paper. The final two sections provide a brief comparison to related work and draw conclusions.</Paragraph> </Section> <Section position="7" start_page="209" end_page="211" type="metho"> <SectionTitle> 2 VERB DETECTION </SectionTitle> <Paragraph position="0"> The technique I developed for finding verbs is based on the Case Filter of Rouvret and Verguaud (1980). The Case Filter is a proposed rule of grammar which, as it applies to English, says that every noun-phrase must appear either immediately to the left of a tensed verb, immediately to the right of a preposition, or immediately to the right of a main verb. Adverbs and adverbial phrases (including days and dates) are ignored for the purposes of case adjacency. A noun-phrase that satisfies the Case Filter is said to &quot;get case&quot; or &quot;have case&quot;, while one that violates it is said to &quot;lack case&quot;. The program judges an open-class word to be a main verb if it is adjacent to a pronoun or proper name that would otherwise lack case. Such a pronoun or proper name is either the subject or the direct object of the verb. Other noun phrases are not used because it is too difficult to determine their right boundaries accurately.</Paragraph> <Paragraph position="1"> The two criteria for evaluating the performance of the main-verb detection technique are efficiency and accuracy. Both were measured using a 2.6 million word corpus for which the Penn Treebank project provides hand-verified tags.</Paragraph> <Paragraph position="2"> Efficiency of verb detection was assessed by running the SF detection module in the normal mode, where verbs were detected using the Case Filter technique, and then running it again with the Penn Tags substituted for the verb detection module. The results are shown in Table 2. Note the five SFs, as tested on 2.6 million words of the Wall Street Journal and controlled by the Penn Treehank's hand-verified tagging the substantial variation among the SFs: for the SFs &quot;direct object&quot; and &quot;direct object & clause&quot; efficiency is roughly 40% and 25%, respectively; for &quot;direct object & infinitive&quot; it drops to about 8%; and for the intransitive SFs it is under 5%.</Paragraph> <Paragraph position="3"> The reason that the transitive SFs fare better is that the direct object gets case from the preceding verb and hence reveals its presence -- intransitive verbs are harder to find. Likewise, clauses fare better than infinitives because their subjects get case from the main verb and hence reveal it, whereas infinitives lack overt subjects. Another obvious factor is that, for every SF listed above except &quot;direct object&quot; two verbs need to be found -- the matrix verb and the complement verb -- if either one is not detected then no observation is recorded.</Paragraph> <Paragraph position="4"> Accuracy was measured by looking at the Penn tag for every word that the system judged to be a verb. Of approximately 5000 verb tokens found by the Case Filter technique, there were 28 disagreements with the hand-verified tags. My program was right in 8 of these cases and wrong in 20, for a 0.24% error-rate beyond the rate using hand-verified tags. Typical disagreements in which my system was right involved verbs that are ambiguous with much more frequent nouns, like mold in &quot;The Soviet Communist Party has the power to shape corporate development and mold it into a body dependent upon it .&quot; There were several systematic constructions in which the Penn tags were right and my system was wrong, including constructions like &quot;We consumers are...&quot; and pseudo-clefts like '~vhat you then do is you make them think .... (These examples are actual text from the Penn corpus.) The extraordinary accuracy of verb detection -- within a tiny fraction of the rate achieved by trained human taggers -- and it's relatively low efficiency are consistent with the priorities laid out in Section 1.2.</Paragraph> <Section position="1" start_page="210" end_page="211" type="sub_section"> <SectionTitle> 2.1 SF DETECTION </SectionTitle> <Paragraph position="0"> The obvious approach to finding SFs like &quot;V NP to V&quot; and &quot;V to V&quot; is to look for occurrences of just those patterns in the training corpus; but the obvious approach fails to address the attachment problem illustrated by example (1) above. The solution is based on the following insights: the clear cases.</Paragraph> <Paragraph position="1"> Rather than take the obvious approach of looking for &quot;V NP to V', my approach is to wait for clear cases like &quot;V PRONOUN to V'. The advantages can be seen by contrasting (3) with (1).</Paragraph> <Paragraph position="2"> (3) a. OK I expected him to eat ice-cream b. * I doubted him to eat ice-cream More generally, the system recognizes linguistic structure using a small finite-state grammar that describes only that fragment of English that is most useful for recognizing SFs. The grammar relies exclusively on closed-class lexical items such as pronouns, prepositions, determiners, and auxiliary verbs.</Paragraph> <Paragraph position="3"> The grammar for detecting SFs needs to distinguish three types of complements: direct objects, infinitives, and clauses. The grammars for each of these are presented in Figure 1. Any open-class word judged to he a verb (see Section 2) and followed immediately by matches for <DO>, <clause>, <infinitives, <DO><clanse>, or <DO><inf> is assigned the corresponding SF. Any word ending in &quot;ly&quot; or</Paragraph> <Paragraph position="5"> an optional element. Any verb followed immediately expressions matching <DO>, <clause>, <infinitive>, <DO> <clause>, or <DO> <infinitive> is assigned the corresponding SF.</Paragraph> <Paragraph position="6"> belonging to a list of 25 irregular adverbs is ignored for purposes of adjacency. The notation &quot;T' follows optional expressions. The category previously-noted-uninflected-verb is special in that it is not fixed in advance -- open-class nonadverbs are added to it when they occur following an unambiguous modal. I This is the only case in which the program makes use of earlier decisions -- literally bootstrapping. Note, however, that ambiguity is possible between mass nouns and uninflected verbs, as in to fish.</Paragraph> <Paragraph position="7"> Like the verb detection algorithm, the SF detection algorithm is evaluated in terms of efficiency and accuracy. The most useful estimate of efficiency is simply the density of observations in the corpus, shown in the first column of Table 3. The million words of the Wall Street Journal accuracy of SF detection is shown in the second 1If there were room to store an unlimited number of uninflected verbs for later reference then the grammar formalism would not be finite-state. In fact, a fixed amount of storage, sufficient to store all the verbs in the language, is allocated. This question is purely academic, however -- a hash-table gives constant-time average performance.</Paragraph> <Paragraph position="8"> column of Table 3. 2 The most common source of error was purpose adjuncts, as in &quot;John quit to pursue a career in finance,&quot; which comes from omitting the in order from &quot;John quit in order to pursue a career in finance.&quot; These purpose adjuncts were mistaken for infinitival complements.</Paragraph> <Paragraph position="9"> The other errors were more sporadic in nature, many coming from unusual extrapositions or other relatively rare phenomena.</Paragraph> <Paragraph position="10"> Once again, the high accuracy and low efficiency are consistent with the priorities of Section 1.2. The throughput rate is currently about ten-thousand words per second on a Sparcstation 2, which is also consistent with the initial priorities. Furthermore, at ten-thousand words per second the current density of observations is not problematic.</Paragraph> </Section> </Section> class="xml-element"></Paper>