File Information
File: 05-lr/acl_arc_1_sum/cleansed_text/xml_by_section/metho/03/p03-1002_metho.xml
Size: 19,969 bytes
Last Modified: 2025-10-06 14:08:14
<?xml version="1.0" standalone="yes"?> <Paper uid="P03-1002"> <Title>Using Predicate-Argument Structures for Information Extraction</Title> <Section position="4" start_page="0" end_page="0" type="metho"> <SectionTitle> BOOLEAN NAMED ENTITY FLAGS [?] A feature set comprising: PHRASAL VERB COLOCATIONS [?] Comprises two features: </SectionTitle> <Paragraph position="0"> [?] pvcSum: the frequency with which a verb is immediately followed by [?] pvcMax: the frequency with which a verb is followed by its any preposition or particle.</Paragraph> <Paragraph position="1"> predominant preposition or particle.</Paragraph> <Paragraph position="2"> [?] neOrganization: set to 1 if an organization is recognized in the phrase [?] neLocation: set to 1 a location is recognized in the phrase [?] nePerson: set to 1 if a person name is recognized in the phrase [?] neMoney: set to 1 if a currency expression is recognized in the phrase [?] nePercent: set to 1 if a percentage expression is recognized in the phrase [?] neTime: set to 1 if a time of day expression is recognized in the phrase [?] neDate: set to 1 if a date temporal expression is recognized in the phrase word from the constituent, different from the head word.</Paragraph> <Paragraph position="3"> [?] CONTENT WORD (cw) [?] Lexicalized feature that selects an informative</Paragraph> </Section> <Section position="5" start_page="0" end_page="0" type="metho"> <SectionTitle> PART OF SPEECH OF HEAD WORD (hPos) [?] The part of speech tag of </SectionTitle> <Paragraph position="0"> the head word.</Paragraph> </Section> <Section position="6" start_page="0" end_page="0" type="metho"> <SectionTitle> PART OF SPEECH OF CONTENT WORD (cPos) [?]The part of speech </SectionTitle> <Paragraph position="0"> tag of the content word.</Paragraph> </Section> <Section position="7" start_page="0" end_page="0" type="metho"> <SectionTitle> NAMED ENTITY CLASS OF CONTENT WORD (cNE) [?] The class of </SectionTitle> <Paragraph position="0"> ferent than the head word. The head words are indicated by the dashed arrows. The content words are indicated by the continuous arrows.</Paragraph> <Paragraph position="1"> on the augmented parser that outputs predicate argument structures. For this reason we used the C5 inductive decision tree learning algorithm (Quinlan, 2002) to implement both the classifier that identifies argument constituents and the classifier that labels arguments with their roles.</Paragraph> <Paragraph position="2"> Our model considers two sets of features: Feature Set 1 (FS1): features used in the work reported in (Gildea and Palmer, 2002) and (Gildea and Jurafsky, 2002) ; and Feature Set 2 (FS2): a novel set of features introduced in this paper. FS1 is illustrated in Figure 3 and FS2 is illustrated in Figure 4.</Paragraph> <Paragraph position="3"> In developing FS2 we used the following observations: null Observation 1: Because most of the predicate arguments are prepositional attachments (PP) or relative clauses (SBAR), often the head word (hw) feature from FS1 is not in fact the most informative word in H1: if phrase type is PP then select the right[?]most child Example: phrase = &quot;in Texas&quot;, cw = &quot;Texas&quot; ifH2: phrase type is SBAR then select the left[?]most sentence (S*) clause Example: phrase = &quot;that occurred yesterday&quot;, cw = &quot;occurred&quot; if thenH3: phrase type is VP if there is a VP child then else select the head word select the left[?]most VP child Example: phrase = &quot;had placed&quot;, cw = &quot;placed&quot; ifH4: phrase type is ADVP then select the right[?]most child not IN or TO Example: phrase = &quot;more than&quot;, cw = &quot;more&quot; ifH5: phrase type is ADJP then select the right[?]most adjective, verb, noun, or ADJP Example: phrase = &quot;61 years old&quot;, cw = &quot;old&quot; H6: for for all other phrase types do select the head word Example: phrase = &quot;red house&quot;, cw = &quot;house&quot; the phrase. Figure 5 illustrates three examples of this situation. In Figure 5(a), the head word of the PP phrase is the preposition in, but June is at least as informative as the head word. Similarly, in Figure 5(b), the relative clause is featured only by the relative pronoun that, whereas the verb occurred should also be taken into account. Figure 5(c) shows another example of an infinitive verb phrase, in which the head word is to, whereas the verb declared should also be considered. Based on these observations, we introduced in FS2 the CONTENT WORD (cw), which adds a new lexicalization from the argument constituent for better content representation. To select the content words we used the heuristics illustrated in Figure 6.</Paragraph> <Paragraph position="4"> After implementing FS1, we noticed that the hw feature was rarely used, and we believe that this happens because of data sparsity. The same was noticed for the cw feature from FS2. Therefore we decided to add two new features, namely the parts of speech of the head word and the content word respectively. These features are called hPos and cPos and are illustrated in Figure 4. Both these features generate an implicit yet simple backoff solution for the lexicalized features HEAD WORD (hw) and CONTENT WORD (cw).</Paragraph> <Paragraph position="5"> Observation 3: Predicate arguments often contain names or other expressions identified by named entity (NE) recognizers, e.g. dates, prices. Thus we believe that this form of semantic information should be introduced in the learning model. In FS2 we added the following features: (a) the named entity class of the content word (cNE); and (b) a set of NE features that can take only Boolean values grouped as</Paragraph> </Section> <Section position="8" start_page="0" end_page="0" type="metho"> <SectionTitle> BOOLEAN NAMED ENTITY FEATURES and defined </SectionTitle> <Paragraph position="0"> in Figure 4. The cNE feature helps recognize the argument roles, e.g. ARGM-LOC and ARGM-TMP, when location or temporal expressions are identified. The Boolean NE flags provide information useful in processing complex nominals occurring in argument constituents. For example, in Figure 2 ARG0 is featured not only by the word traders but also by ORGANIZATION, the semantic class of the name Big Board.</Paragraph> <Paragraph position="1"> Observation 4: Predicate argument structures are recognized accurately when both predicates and arguments are correctly identified. Often, predicates are lexicalized by phrasal verbs, e.g. put up, put off. To identify correctly the verb particle and capture it in the structure of predicates instead of the argument structure, we introduced two collocation features that measure the frequency with which verbs and succeeding prepositions cooccurr in the corpus. The features are pvc-Sum and pvcMax and are defined in Figure 4.</Paragraph> <Section position="1" start_page="0" end_page="0" type="sub_section"> <SectionTitle> 2.3 The Experiments </SectionTitle> <Paragraph position="0"> The results presented in this paper were obtained by training on Proposition Bank (PropBank) release 2002/7/15 (Kingsbury et al., 2002). Syntactic information was extracted from the gold-standard parses in TreeBank Release 2. As named entity information is not available in PropBank/TreeBank we tagged the training corpus with NE information using an open-domain NE recognizer, having 96% F-measure on the MUC61 data. We reserved section 23 of Prop-Bank/TreeBank for testing, and we trained on the rest. Due to memory limitations on our hardware, for the argument finding task we trained on the first evaluation exercises in the 90s. Starting with MUC6 named entity data was available.</Paragraph> <Paragraph position="1"> for the role assignment task on the first 75 KB of argument constituents (about 60% of PropBank annotations). null Table 1 shows the results obtained by our inductive learning approach. The first column describes the feature sets used in each of the 7 experiments performed. The following three columns indicate the precision (P), recall (R), and F-measure (a0a2a1 )2 obtained for the task of identifying argument constituents. The last column shows the accuracy (A) for the role assignment task using known argument constituents. The first row in Table 1 lists the results obtained when using only the FS1 features.</Paragraph> <Paragraph position="2"> The next five lines list the individual contributions of each of the newly added features when combined with the FS1 features. The last line shows the results obtained when all features from FS1 and FS2 were used.</Paragraph> <Paragraph position="3"> Table 1 shows that the new features increase the argument identification F-measure by 3.61%, and the role assignment accuracy with 4.29%. For the argument identification task, the head and content word features have a significant contribution for the task precision, whereas NE features contribute significantly to the task recall. For the role assignment task the best features from the feature set FS2 are the content word features (cw and cPos) and the Boolean NE flags, which show that semantic information, even if minimal, is important for role classification. Surprisingly, the phrasal verb collocation features did not help for any of the tasks, but they were useful for boosting the decision trees. Decision tree learning provided by C5 (Quinlan, 2002) has built in support for boosting. We used it and obtained improvements for both tasks. The best F-measure obtained for argument constituent identification was 88.98% in the fifth iteration (a 0.76% improvement). The best accuracy for role assignment was 83.74% in the eight iteration (a 0.69% improvement)3. We further analyzed the boosted trees and noticed that phrasal verb collocation features were mainly responsible for the improvements. This is the rationale for including them in the FS2 set.</Paragraph> <Paragraph position="4"> We also were interested in comparing the results</Paragraph> <Paragraph position="6"> These results, listed also on the last line of Table 2, differ from those in Table 1 because they were produced after the boosting took place.</Paragraph> </Section> <Section position="2" start_page="0" end_page="0" type="sub_section"> <SectionTitle> learning models </SectionTitle> <Paragraph position="0"> of the decision-tree-based method against the results obtained by the statistical approach reported in (Gildea and Palmer, 2002). Table 2 summarizes the results. (Gildea and Palmer, 2002) report the results listed on the first line of Table 2. Because no F-scores were reported for the argument identification task, we re-implemented the model and obtained the results listed on the second line. It looks like we had some implementation differences, and our results for the argument role classification task were slightly worse. However, we used our results for the statistical model for comparing with the inductive learning model because we used the same feature extraction code for both models. Lines 3 and 4 list the results of the inductive learning model with boosting enabled, when the features were only from FS1, and from FS1 and FS2 respectively. When comparing the results obtained for both models when using only features from FS1, we find that almost the same results were obtained for role classification, but an enhancement of almost 13% was obtained when recognizing argument constituents. When comparing the statistical model with the inductive model that uses all features, there is an enhancement of 17.12% for argument identification and 4.87% for argument role recognition.</Paragraph> <Paragraph position="1"> Another significant advantage of our inductive learning approach is that it scales better to un- null in Gildea and Jurafsky (2002) uses predicate lexical information at most levels in the probability lattice, hence its scalability to unknown predicates is limited. In contrast, the decision tree approach uses predicate lexical information only for 5% of the branching decisions recorded when testing the role assignment task, and only for 0.01% of the branching decisions seen during the argument constituent identification evaluation.</Paragraph> </Section> </Section> <Section position="9" start_page="0" end_page="0" type="metho"> <SectionTitle> 3 The IE Paradigm </SectionTitle> <Paragraph position="0"> Figure 7(a) illustrates an IE architecture that employs predicate argument structures. Documents are processed in parallel to: (1) parse them syntactically, and (2) recognize the NEs. The full parser first performs part-of-speech (POS) tagging using transformation based learning (TBL) (Brill, 1995). Then non-recursive, or basic, noun phrases (NPB) are identified using the TBL method reported in (Ngai and Florian, 2001). At last, the dependency parser presented in (Collins, 1997) is used to generate the full parse. This approach allows us to parse the sentences with less than 40 words from TreeBank section 23 with an F-measure slightly over 85% at an average of 0.12 seconds/sentence on a 2GHz Pentium IV computer.</Paragraph> <Paragraph position="1"> The parse texts marked with NE tags are passed to a module that identifies entity coreference in documents, resolving pronominal and nominal anaphors and normalizing coreferring expressions. The parses are also used by a module that recognizes predicate argument structures with any of the methods described in Section 2.</Paragraph> <Paragraph position="2"> For each templette modeling a different domain a mapping between predicate arguments and templette slots is produced. Figure 8 illustrates the mapping produced for two Event99 do-</Paragraph> </Section> <Section position="10" start_page="0" end_page="0" type="metho"> <SectionTitle> LOCATION LOCATION </SectionTitle> <Paragraph position="0"> mains. The &quot;market change&quot; domain monitors changes (AMOUNT CHANGE) and current values (CURRENT VALUE) for financial instruments (IN-STRUMENT). The &quot;death&quot; domain extracts the description of the person deceased (DECEASED), the manner of death (MANNER OF DEATH), and, if applicable, the person to whom the death is attributed</Paragraph> </Section> <Section position="11" start_page="0" end_page="0" type="metho"> <SectionTitle> (AGENT OF DEATH). </SectionTitle> <Paragraph position="0"> To produce the mappings we used training data that consists of: (1) texts, and (2) their corresponding filled templettes. Each templette has pointers back to the source text similarly to the example presented in Figure 1. When the predicate argument structures were identified, the mappings were collected as illustrated in Figure 9. Figure 9(a) shows an interesting aspect of the mappings. Although the role classification of the last argument is incorrect (it should have been identified as ARG4), it is mapped into the CURRENT-VALUE slot. This shows how the mappings resolve incorrect but consistent classifications. Figure 9(b) shows the flexibility of the system to identify and classify constituents that are not close to the predicate phrase (ARG0). This is a clear ad- null vantage over the FSA-based system, which in fact missed the AGENT-OF-DEATH in this sentence. Because several templettes might describe the same event, event coreference is processed and, based on the results, templettes are merged when necessary.</Paragraph> <Paragraph position="1"> The IE architecture in Figure 7(a) may be compared with the IE architecture with cascaded FSA represented in Figure 7(b) and reported in (Surdeanu and Harabagiu, 2002). Both architectures share the same NER, coreference and merging modules. Specific to the FSA-based architecture are the phrasal parser, which identifies simple phrases such as basic noun or verb phrases (some of them domain specific), the combiner, which builds domain-dependent complex phrases, and the event recognizer, which detects the domain-specific Subject-Verb-Object (SVO) patterns. An example of a pattern used by the FSA-based architecture is: a0 DEATH-CAUSE KILL-VERB PERSONa1 , where DEATH-CAUSE may identify more than 20 lexemes, e.g. wreck, catastrophe, malpractice, and more than 20 verbs are KILL-VERBS, e.g. murder, execute, behead, slay. Most importantly, each pattern must recognize up to 26 syntactic variations, e.g. determined by the active or passive form of the verb, relative subjects or objects etc. Predicate argument structures offer the great advantage that syntactic variations do not need to be accounted by IE systems anymore.</Paragraph> <Paragraph position="2"> Because entity and event coreference, as well as templette merging will attempt to recover from partial patterns or predicate argument recognitions, and our goal is to compare the usage of FSA patterns versus predicate argument structures, we decided to disable the coreference and merging modules. This explains why in Figure 7 these modules are repre- null or predicate argument structures) matched sented with dashed lines.</Paragraph> </Section> <Section position="12" start_page="0" end_page="0" type="metho"> <SectionTitle> 4 Experiments with The Integration of </SectionTitle> <Paragraph position="0"> Predicate Argument Structures in IE To evaluate the proposed IE paradigm we selected two Event99 domains: &quot;market change&quot;, which tracks changes in stock indexes, and &quot;death&quot;, which extracts all manners of human deaths. These domains were selected because most of the domain information can be processed without needing entity or event coreference. Moreover, one of the domains (market change) uses verbs commonly used in Prop-Bank/TreeBank, while the other (death) uses relatively unknown verbs, so we can also evaluate how well the system scales to verbs unseen in training.</Paragraph> <Paragraph position="1"> Table 3 lists the F-scores for the two domains.</Paragraph> <Paragraph position="2"> The first line of the Table lists the results obtained by the IE architecture illustrated in Figure 7(a) when the predicate argument structures were identified by the statistical model. The next line shows the same results for the inductive learning model. The last line shows the results for the IE architecture in Figure 7(b). The results obtained by the FSA-based IE were the best, but they were made possible by hand-crafted patterns requiring an effort of 10 person days per domain. The only human effort necessary in the new IE paradigm was imposed by the generation of mappings between arguments and templette slots, accomplished in less than 2 hours per domain, given that the training templettes are known. Additionally, it is easier to automatically learn these mappings than to acquire FSA patterns.</Paragraph> <Paragraph position="3"> Table 3 also shows that the new IE paradigm performs better when the predicate argument structures are recognized with the inductive learning model.</Paragraph> <Paragraph position="4"> The cause is the substantial difference in quality of the argument identification task between the two models. The Table shows that the new IE paradigm with the inductive learning model achieves about 90% of the performance of the FSA-based system for both domains, even though one of the domains uses mainly verbs rarely seen in training (e.g. &quot;die&quot; appears 5 times in PropBank).</Paragraph> <Paragraph position="5"> Another way of evaluating the integration of predicate argument structures in IE is by comparing the number of events identified by each architecture. Table 4 shows the results. Once again, the new IE paradigm performs better when the predicate argument structures are recognized with the inductive learning model. More events are missed by the statistical model which does not recognize argument constituents as well the inductive learning model.</Paragraph> </Section> class="xml-element"></Paper>