<?xml version="1.0" standalone="yes"?>
<Paper uid="W01-1206">
  <Title>Answer Mining from On-Line Documents</Title>
  <Section position="4" start_page="0" end_page="0" type="metho">
    <SectionTitle>
ILANDMARK Reimssee
</SectionTitle>
    <Paragraph position="0"> What could I see in Reims? structures of ET1 and ET2 can be mapped one into another. The mapping is produced by unifying the two structures when lexical and semantic alternations are allowed. Possible lexical alternations are synonyms or morphological alternations. Semantic alternations consist of hypernyms, entailments or paraphrases. The unifying mapping of ET1 and ET2 shows that the two questions are equivalent only when I refers to a visitor; other readings of ET2 being possible when the referent is an investigator or a politician. In each of the other readings, the answer type of the question would be different. The unifying mapping of ET1 and ET2 is: ReimsI --&gt;tourists see/visitLANDMARK Similarly, a pair of equivalent answers is recognized when lexical and semantic alternations of the concepts are allowed. This observation is crucial for answer mining because: 1. it establishes the dependency relations as the basic processing level for Q/A; and 2. it defines the search space based on alternations of the question and answer concepts.</Paragraph>
    <Paragraph position="1"> Consequently, lexical and semantic alternations are incorporated as feedback loops in the architecture of open-domain Q/A systems, as illustrated in Figure 2.</Paragraph>
    <Paragraph position="2"> To locate answers, text passages are retrieved based on keywords assembled from the question dependency structure. At the time of the query, it is unknown which keywords can be unified with answer dependencies. However, the relevance of the query is determined by the number of resulting passages. If too many passages are generated, the query was too broad, thus is needs a specialization by adding a new keyword. If too few passages were retrieved the query was too specific, thus one keyword needs to be dropped.</Paragraph>
    <Paragraph position="3"> The relevance feedback based on the number of retrieved passages ends when no more keywords can be added or dropped. After this, the unifications of the question and answer dependencies  by unifications are added to the list of keywords, making possible the retrieval of new, unseen text passages, as illustrated in Figure 2.</Paragraph>
    <Paragraph position="4"> The unification of dependency structures allows erroneous answers when the resulting mapping is a sparse graph. To justify the correctness of the answer an abductive proof backchaining from the answer to the question must be produced. Such abductive mechanisms are detailed in (Harabagiu et al., 2000). Moreover, the proof relies on lexico-semantic knowledge available from WordNet as well as rapidly formated knowledge bases generated by mechanisms described in (Chaudri et al., 2000). The justification process brings forward semantic alternations that are added to the list of keywords, the feedback destination of all loops represented in Figure 2.</Paragraph>
    <Paragraph position="5"> Mining the exact answer does not always end after extracting the answer type from a correct text snippet because often they result only in partial answers that need to be fused together. The fusion mechanisms are dictated by the answer type.</Paragraph>
    <Section position="1" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
Question Complexity
</SectionTitle>
      <Paragraph position="0"> Open-Domain natural language questions can also be of different complexity levels. Generally, the test questions used in the TREC evaluations were qualified as fact-based questions (cf. (Voorhees and Tice, 2000)) as they mainly were short inquiries about attributes or definitions of some entity or event. Table 1 lists a sample of TREC test questions.</Paragraph>
      <Paragraph position="1"> The TREC test set did not include any question Where is Romania located? Europe Who wrote &amp;quot;Dubliners&amp;quot;? James Joyce What is the wingspan of a condor? 9 feet What is the population of Japan? 120 million What king signed the Magna Carta? King John Name a flying mammal. bat  that can be modeled as Information Extraction (IE) task. Typically, IE templates model queries regarding who did an event of interest, what was produced by that event, when and where and eventually why. The event of interest is a complex event, like terrorism in Latin America, joint ventures or management successions. An example of template-modeled question is: What management successions occurred at IBM in 1999? In addition, questions may also ask about developments of events or trends that are usually answered by a text summary. Since data producing these summaries can be sourced in different documents, summary fusion techniques as proposed in (Radev and McKeown, 1998) can be employed. Template-based questions and summaryasking inquiries cover most of the classes of question complexity proposed in (Moldovan et al., 2000). Although the topic of natural language open-domain question complexity needs further study, we consider herein the following classes of questions: a2 Class 1: Questions inquiring about entities, events, entity attributes (including number), event themes, event manners, event conditions and event consequences.</Paragraph>
      <Paragraph position="2"> a2 Class 2: Questions modeled by templates, including questions that focus only on one of the template slots (e.g. &amp;quot;What managers were promoted last year at Microsoft?&amp;quot;).</Paragraph>
      <Paragraph position="3"> a2 Class 3: Questions asking for a summary that is produced by fusing template-based information from different sources (e.g. &amp;quot;What happened after the Titanic sunk?&amp;quot;).</Paragraph>
      <Paragraph position="4"> Since (Radev and McKeown, 1998) describes the summary fusion mechanisms, Class 3 of questions can be reduced in this paper to Class 2, which deals with the processing of the template.</Paragraph>
    </Section>
  </Section>
  <Section position="5" start_page="0" end_page="0" type="metho">
    <SectionTitle>
3 A Model of Answer Types
</SectionTitle>
    <Paragraph position="0"> This section describes a knowledge-based model of open-domain natural language answer types (ATs). In particular we formally define the answer type through a quadruple a0a2a1a4a3a6a5CATEGORY, DEPENDENCY, NUMBER, FORMATa7.</Paragraph>
    <Paragraph position="1"> The CATEGORY is defined as one of the following possibilities:  1. one of the tops of a predefined ANSWER TAXONOMY or one of its nodes; 2. DEFINITION; 3. TEMPLATE; or 4. SUMMARY.</Paragraph>
    <Paragraph position="2"> For expert Q/A systems, this list of categories can be extended. The DEPENDENCY is defined as the  question dependency structure when the CATEGORY belongs to the ANSWER TAXONOMY or is a DEFINITION. Otherwise it is a template automatically generated. The NUMBER is a flag indicating whether the answer should contain a single datum or a list of elements. The FORMAT defines the text span of the exact answer. For example, if the CATEGORY is DIMENSION, the FORMAT is</Paragraph>
    <Paragraph position="4"> The ANSWER TAXONOMY was created in three steps: Step 1 We devise a set of top categories modeled after the semantic domains encoded in the Word-Net database, which contains 25 noun categories and 15 verb categories. The top of each WordNet hierarchy corresponding to every semantic category was manually inspected to select the most representative nodes and add them to the tops of he ANSWER TAXONOMY. Furthermore we have added open semantic categories corresponding to named entities. For example Table 2 lists the named entity categories we have considered in our experiments. Many of the tops of the ANSWER TAXONOMY are further categorized, as illustrated in Figure 3. In total, we have considered 33 concepts as tops of the taxonomy.</Paragraph>
    <Paragraph position="5">  many mapping of the Named Entity categories in the tops of the ANSWER TAXONOMY. Figure 4 illustrates some of the mappings.</Paragraph>
    <Paragraph position="6"> date time organization city product price country money human disease phone number continent percent province other location plant mammal alphabet airport code game bird reptile university dog breed number quantity landmark dish</Paragraph>
  </Section>
  <Section position="6" start_page="0" end_page="0" type="metho">
    <SectionTitle>
ATIONALITY
</SectionTitle>
    <Paragraph position="0"> of the trip from...? What is the duration of an active volcano get?</Paragraph>
  </Section>
  <Section position="7" start_page="0" end_page="0" type="metho">
    <SectionTitle>
EGREED TEMPERATURE DURATION COUNT SPEED DIMENSION
OCATIONL
</SectionTitle>
    <Paragraph position="0"> What is the wingspan of a condor? in diameter? How big is our galaxyHow hot does the inside  Net subherarchies. Figure 5 illustrates a fragment of the ANSWER TAXONOMY comprising several WordNet subhierarchies.</Paragraph>
  </Section>
  <Section position="8" start_page="0" end_page="0" type="metho">
    <SectionTitle>
4 Answer Recognition and Extraction
</SectionTitle>
    <Paragraph position="0"> In this section we show how, given a question and its dependency structure, we can recognize its answer type and consequently extract the exact answer. Here we describe four representative cases.</Paragraph>
    <Paragraph position="1"> Case 1: The CATEGORY of the answer type is DEFINITION when the question can be matched by one of the following patterns: (Q-P1):What a0 isa1area2a4a3 phrase to definea5 ? (Q-P2):What is the definition of a3 phrase to definea5 ? (Q-P3):Who a0 isa1wasa1area1werea2a6a3 person name(s)a5 ? The format of the DEFINITION answers is similarly dependent on a set of patterns, determined as the head of the a8 Answer phrasea9 :  tion indicates that a special instance of a concept is sought. The cues are given either by the presence of words kind, type, name or by the question stems what or which connected to the object of a verb. Table 3 lists a set of such questions and their corresponding answers. In this case the answer type is given by the subhierarchy defined by the node from the dependency structure whose adjunct is either kind, type, name or the question stem. In this situation the CATEGORY does not belong to the top of the ANSWER TAXONOMY, but it is rather dynamically created by the interpretation of the dependency graph.</Paragraph>
    <Paragraph position="2"> For example, the dynamic CATEGORY bridge, generated for Q204 from Table 3, contains 14 member instances, including viaduct, rope bridge and suspension bridge. Similarly, question Q581 generates a dynamic CATEGORY flower, with 470 member instances, comprising orchid, petunia and sunflower. For dynamic categories all member instances are searched in the retrieved passages during answer extraction to detect candidate answers.</Paragraph>
    <Paragraph position="3"> Case 3: In all other cases, the concept related to the question stem in the question dependency graph is searched through the ANSWER TAXON-OMY, returning the answer type as the top of it hierarchy. Figure 5 illustrates several questions and their answer type CATEGORY.</Paragraph>
    <Paragraph position="4"> Case 4: Whenever the semantic dependencies of several correct answers can be mapped one into another, we change the CATEGORY of the answer type into TEMPLATE. The slots of the actual template are determined by a three step procedure, that we illustrate with a walk-through example corresponding to the question What management successions occurred at IBM in 1999?: Step 1: For each pair of extracted candidate Q204: What type of bridge is the Golden Gate Bridge? Answer: the Seto Ohashi Bridge, consisting of six suspension bridges in the style of Golden Gate Bridge. Q267: What is the name for clouds that produce rain? Answer: Acid rain in Cheju Island and the Taean peninsula is carried by rain clouds from China. Q503: What kind of sports team is the Buffalo Sabres? Answer: Alexander Mogilny hopes to continue his hockey career with the NHL's Buffalo Sabres. Q581: What flower did Vincent Van Gogh paint? Answer: In March 1987, van Gogh's &amp;quot;Sunflowers&amp;quot; sold for $39.9 million at Christie's in London  answers unify the dependency graphs and find common generalizations whenever possible. Figure 6(a) illustrates some of the mappings.</Paragraph>
    <Paragraph position="5"> Step 2: Identify across mappings the common categories and the trigger-words that were used as keywords. In Figure 6(a) the trigger words are boldfaced.</Paragraph>
    <Paragraph position="6"> Step 3: Collect all common categories in a template and use their names as slots. Figure 6(b) illustrates the resulting template.</Paragraph>
    <Paragraph position="7"> This procedure is a reverse-engineering of the mechanisms used generally in Information Extraction (IE), where given a template, linguistic patterns are acquired to identify the text fragments having relevant information. In the case of answer mining, the relevant text passages are known. The dependency graphs help finding the linguistic rules and are generalized in a template.</Paragraph>
    <Paragraph position="8"> To be able to generate the template we also need to have a way of extracting the text where the answer dependencies are detected. For this purpose we have designed a method that employs a simple machine learning mechanism: the perceptron. For each text passage retrieved by the keyword-based query we define the following seven features: a0a2a1a4a3a6a5a8a7a4a9 the number of question words matched in the same phrase as the answer type CATEGORY; a0a10a1a4a3a6a5a8a7a11a7 the number of question words matched in the same sentence as the answer type CATEGORY; a0a12a1a4a3a6a5a14a13a15a9 : a flag set to 1 if the answer type CATEGORY is followed by a punctuation sign, and set to 0 otherwise; a0a16a1a4a3a6a5a8a17a19a18a21a20a23a22 : the number of question words matches separated from the answer type CATEGORY by at most three words and one comma; a0a24a1a4a3a6a5a8a7a11a22a25a7 : the number of question words occurring in the same order in the answer text as in the question; a0a26a1a4a3a6a5a8a27a28a20a23a22 : the average distance from the answer type CATEGORY to any of the question word matches; a0a29a1a4a3a6a5a8a30a10a31a32a22 : the number of question words matched in the answer text.</Paragraph>
    <Paragraph position="9"> To train the perceptron we annotated the correct answers of 200 of the TREC test questions. Given a pair of answers, in which one of the answers is correct, we compute a relative comparison score using the formula:  The perceptron learns the seven weights as well as the value of the threshold used for future tests on the remaining 693 TREC questions. Whenever the relative score is larger than the threshold, a passage is extracted as a candidate answer. In our experiments, the performance of the perceptron surpassed the performance of decision trees for answer extraction.</Paragraph>
  </Section>
class="xml-element"></Paper>