<?xml version="1.0" standalone="yes"?>
<Paper uid="X98-1016">
  <Title>Transforming Examples into Patterns for Information Extraction</Title>
  <Section position="4" start_page="99" end_page="99" type="metho">
    <SectionTitle>
4 Example-based Acquisition
</SectionTitle>
    <Paragraph position="0"/>
    <Section position="1" start_page="99" end_page="99" type="sub_section">
      <SectionTitle>
4.1 Objective
</SectionTitle>
      <Paragraph position="0"> Consider a situation where the developer has found a salient text segment and proceeds to extend the IE system to extract the proper information from it.</Paragraph>
      <Paragraph position="1"> Figure 3 shows a (paraphrased) text segment from the MUC-6 development corpus, with the corresponding extracted event, in the form of a database record. We will use this example to illustrate our methodology.</Paragraph>
      <Paragraph position="2"> In our earlier system (as in most other IE systems), upon finding a candidate example, the developer had to construct a pattern capable of capturing the example. Such a pattern consists of two parts: * the precondition, which seeks to match an active clause beginning with a np of type &amp;quot;company&amp;quot;, followed by a verb group (vg) of class &amp;quot;appoint&amp;quot;, followed by a np of class &amp;quot;person&amp;quot;, etc.; * the action which fires when the pattern matches, and prescribes the operations to be performed on the sentence fragments and the logical form.</Paragraph>
      <Paragraph position="3"> Figure 4 shows an excerpt from the pattern code; it is written in Common Lisp, with the precondition specified using a special &amp;quot;pattern language&amp;quot;. Clearly, this method of development is quite time-consuming and error-prone.</Paragraph>
      <Paragraph position="4"> Instead, we now employ a kind of a &amp;quot;bootstrapping&amp;quot; procedure: the system allows the user to introduce a new example and apply to it the patterns that the system has acquired so far. This produces a partial analysis, and builds LFs for the analyzable constituents of the example text. The user then specifies how these LFs, or their sub-parts, combine to yield the LF for the entire example.</Paragraph>
    </Section>
    <Section position="2" start_page="99" end_page="99" type="sub_section">
      <SectionTitle>
4.2 Acquiring Preconditions
</SectionTitle>
      <Paragraph position="0"> To illustrate this method, we first show how the system acquires the pattern for analyzing the portion of our example shown in figure 6. This is a complex NP made up of a series of nouns, names, and other lexical items. The crucial point is that a basic system, which has not been specialized for any domain, will analyze this reduced example as in figure 5 by dint of its built-in patterns for named entities. 7 The analysis also produces the LF entities for each boxed segment. This information is sufficient for the system to propose a precondition for a candidate pattern: s</Paragraph>
      <Paragraph position="2"> The system then initiates an interaction with the user in which s/he can operate on the components, modifying them or labeling them as optional (indicated below by a following question mark), to complete the precondition of the pattern: \[n(C-company) 's\]? \[n(C-city)-based\]? n(Ccompany) n(operation)?</Paragraph>
    </Section>
    <Section position="3" start_page="99" end_page="99" type="sub_section">
      <SectionTitle>
4.3 Acquiring Actions
</SectionTitle>
      <Paragraph position="0"> Now the user specifies, again by example, what action is to be performed when the precondition of the pattern matches. S/he can select the new type of event 7The customization tool supports two methods for analyzing specific noun group structures. The approach described here involves the creation of a semantically-specific noun group pattern. Alternatively, the phrase can first be analyzed by the general, syntactic noun group pattern, with the resulting LF then restructured by a semantically-specific pattern.</Paragraph>
      <Paragraph position="1"> SFor purposes of presentation, we have simplified the form of these patterns to emphasize the parallel with the clause patterns. In the current implementation, each pattern element would involve a conjunction of tests for the syntactic type (n) and the semantic class (C-company, etc.).</Paragraph>
    </Section>
  </Section>
  <Section position="5" start_page="99" end_page="99" type="metho">
    <SectionTitle>
LF
</SectionTitle>
    <Paragraph position="0"> or entity to be created, and indicate how the matched constituents (LFs) discovered in the example are to function in the new event. Alternatively, s/he may designate one of the generated entities as the &amp;quot;head&amp;quot; entity (or the matrix entity) for the complex phrase, and designate the remaining entities as subordinate to the matrix entity, i.e., as standing in some semantic relation to it. To accomplish this, the user can drag-and-drop a subordinate entity into the appropriate slot in the matrix entity (in a simple GUI environment); the slot serves to indicate the relationship of the subordinate entity to the matrix; (see figure 6).</Paragraph>
    <Paragraph position="1"> The precondition and the action together now constitute a complete pattern which matches a complex NP and produces a LF with relations.</Paragraph>
    <Section position="1" start_page="99" end_page="99" type="sub_section">
      <SectionTitle>
4.4 Semantic Generalization
</SectionTitle>
      <Paragraph position="0"> Consider the final, optional constituent in the pre-condition of the preceding pattern, n(operation). We would like to broaden the coverage of the pattern, so that it could match any semantically similar noun in that position; in particular, it should also match &amp;quot;concern&amp;quot;, &amp;quot;outfit&amp;quot;, etc. To this end, our system allows the user to gather semantic concepts in an inheritance hierarchy. For example, s/he can gather all these and more lexemes under the same semantic class, called, e.g., C-co-descrip. Similarly, the classes C-city for city names and C-state for state names would be gathered under a concept C-location. The GUI tools then allow the user to perform semantic generalization on the individual constituents of the pattern's precondition; its final form becomes:</Paragraph>
      <Paragraph position="2"> The semantic hierarchy is scenario-specific. It is built up dynamically through tools that draw on pre-existing domain-independent hierarchies, such as WordNet, as well as domain-specific word similarity measures and co-occurrence statistics \[4\].</Paragraph>
      <Paragraph position="3"> By a similar process, we can now acquire a clausal pattern from the example in figure 3 at the beginning of this *section. The system proposes the precondition: null np(C-company) vg(C-appoint) np(C-person) np(president) Applying semantic generalization to the last constituent yields: np(C-company) vg(C-appoint) np(C-person) np(C-title) where C-title is a semantic class that gathers all corporate titles. The user can now fill the slots in the LF for the event as in figure 7.</Paragraph>
    </Section>
  </Section>
  <Section position="6" start_page="99" end_page="101" type="metho">
    <SectionTitle>
5 Meta-rules
</SectionTitle>
    <Paragraph position="0"> Consider the following variant of the original example: null ... George Garrick, an avowed anticapitalist, was appointed yesterday president of Information Resources Inc., ... The basic pattern for an active clause, which we acquired in the preceding section, will not match this paraphrase. There are two essential kinds of variations here:  * syntactic transformations; the system needs several related patterns, which capture the corresponding passive clause, relative clause, and other syntactic variants of the example.</Paragraph>
    <Paragraph position="1"> * optional, semantically irrelevant modifiers, e.g., sentence adjuncts, appositions, etc., as exemplified by the italicized segments above.</Paragraph>
    <Paragraph position="2"> The user could, of course, provide transformed examples, build patterns individually for each transformation of the original, and insert the optional modifiers to make the patterns as general as possible. However, it is clear that this naive approach quickly leads to a proliferation of patterns with which the user is directly concerned. Instead, we have introduced a meta-rule mechanism: after a pattern is accepted, the system generates all related generalizations of the pattern automatically. 9 For example, from the active clause pattern above, a passivizing meta-rule will produce the precondition: np(C-person) rn? sa? pass-vg(C-appoint) sa? np(C-title) \[by np(C-company)\]? lO The resulting pattern will match the variant example, and produce the correct event LF. To maximize coverage, the system should contain meta-rules for all clausal variants, including nominalizations; similar meta-rules can be provided to generalize noun-phrase patterns, as discussed in section 4.2.</Paragraph>
  </Section>
  <Section position="7" start_page="101" end_page="101" type="metho">
    <SectionTitle>
6 Discussion
</SectionTitle>
    <Paragraph position="0"> We have described a comprehensive methodology for acquiring patterns from examples and automatically expanding their coverage. Other IE systems employ variants of example-based pattern acquisition. One system, developed at University of Massachusetts at Amherst, \[10\], used unsupervised training to learn patterns from the MUC training corpus. However, unsupervised learning can degrade in the face of \[1\] sparse data; the UMass system seemed to require one more order of magnitude of training data than was available in MUC-6. The HASTEN system, developed by SRA \[7\], used a somewhat different example-based approach: they seek to broaden coverage by allowing statistically approximate matches, a strategy that lacks a syntactic basis, and may result in overgeneration. \[2\]  right noun-phrase modifiers, sa is a sentence adjunct, and pass-vg is a passive verb group.</Paragraph>
    <Paragraph position="1"> The presented methodology has been fully implemented as a set of tools that complement our core information extraction engine, and has been tested on three different scenarios. One of the scenarios was successfully implemented by a computational linguist who interacted with the system exclusively by means of the tools, and had no familiarity with the system internals.</Paragraph>
    <Paragraph position="2"> Our experience also suggests areas of improvement, which we are currently pursuing. One important question is: where do examples come from? We seek to shift the burden of inventing the examples from the developer to the system. In response to these problems we are building tools that will help the user surf the corpus to help discover patterns.</Paragraph>
  </Section>
class="xml-element"></Paper>