<?xml version="1.0" standalone="yes"?>
<Paper uid="P03-1044">
  <Title>Counter-Training in Discovery of Semantic Patterns</Title>
  <Section position="3" start_page="0" end_page="0" type="intro">
    <SectionTitle>
2 Background
</SectionTitle>
    <Paragraph position="0"/>
    <Section position="1" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
2.1 Unsupervised Pattern Learning
</SectionTitle>
      <Paragraph position="0"> We outline those aspects of the prior work that are relevant to the algorithm developed in our presentation. null a0 We are given an IE scenario a1 , e.g., &amp;quot;Management Succession&amp;quot; (as in MUC-6). We have a raw general news corpus for training, i.e., an unclassified and un-tagged set of documents a2a4a3 . The problem is to find a good set of patterns in a2 a3 , which cover events relevant to a1 .</Paragraph>
      <Paragraph position="1"> We presuppose the existence of two generalpurpose, lower-level language tools--a name recognizer and a parser. These tools are used to extract all potential patterns from the corpus.</Paragraph>
      <Paragraph position="2"> a0 The user provides a small number of seed patterns for a1 . The algorithm uses the corpus to iteratively bootstrap a larger set of good patterns for a1 . a0 The algorithm/learner achieves this bootstrapping by utilizing the duality between the space of documents and the space of patterns: good extraction patterns select documents relevant to the chosen scenario; conversely, relevant documents typically contain more than one good pattern. This duality drives the bootstrapping process.</Paragraph>
      <Paragraph position="3"> a0 The primary aim of the learning is to train a strong recognizer a5 for a1 ; a5 is embodied in the set of good patterns. However, as a result of training a5 , the procedure also produces the set a2a7a6 a8a10a9a12a11 of documents that it deems relevant to a1 --the documents selected by a5 .</Paragraph>
      <Paragraph position="4"> a0 Evaluation: to evaluate the quality of discovered patterns, (Riloff, 1996) describes a direct evaluation strategy, where precision of the patterns resulting from a given run is established by manual review. (Yangarber et al., 2000) uses an automatic but indirect evaluation of the recognizer a5 : they retrieve a test sub-set a2 a6a13 a9a12a11 a13a15a14 a2 a3 from the training corpus and manually judge the relevance of every document in a2a16a6a13 a9a17a11 a13 ; one can then obtain standard IR-style recall and precision scores for a2 a6a8a10a9a12a11 relative to a2 a6a13 a9a17a11 a13 . In presenting our results, we will discuss both kinds of evaluation.</Paragraph>
      <Paragraph position="5"> The recall/precision curves produced by the indirect evaluation generally reach some level of recall at which precision begins to drop. This happens because at some point in the learning process the algorithm picks up patterns that are common in a1 , but are not sufficiently specific to a1 alone. These patterns then pick up irrelevant documents, and precision drops.</Paragraph>
      <Paragraph position="6"> Our goal is to prevent this kind of degradation, by helping the learner stop when precision is still high, while achieving maximal recall.</Paragraph>
    </Section>
    <Section position="2" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
2.2 Related Work
</SectionTitle>
      <Paragraph position="0"> We briefly mention some of the unsupervised methods for acquiring knowledge for NL understanding, in particular in the context of IE. A typical architecture for an IE system includes knowledge bases (KBs), which must be customized when the system is ported to new domains. The KBs cover different levels, viz. a lexicon, a semantic conceptual hierarchy, a set of patterns, a set of inference rules, a set of logical representations for objects in the domain.</Paragraph>
      <Paragraph position="1"> Each KB can be expected to be domain-specific, to a greater or lesser degree.</Paragraph>
      <Paragraph position="2"> Among the research that deals with automatic acquisition of knowledge from text, the following are particularly relevant to us. (Strzalkowski and Wang, 1996) proposed a method for learning concepts belonging to a given semantic class. (Riloff and Jones, 1999; Riloff, 1996; Yangarber et al., 2000) present different combinations of learners of patterns and concept classes specifically for IE.</Paragraph>
      <Paragraph position="3"> In (Riloff, 1996) the system AutoSlog-TS learns patterns for filling an individual slot in an event template, while simultaneously acquiring a set of lexical elements/concepts eligible to fill the slot. AutoSlog-TS, does not require a pre-annotated corpus, but does require one that has been split into subsets that are relevant vs. non-relevant subsets to the scenario.</Paragraph>
      <Paragraph position="4"> (Yangarber et al., 2000) attempts to find extraction patterns, without a pre-classified corpus, starting from a set of seed patterns. This is the basic unsupervised learner on which our approach is founded; it is described in the next section.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>