<?xml version="1.0" standalone="yes"?>
<Paper uid="P01-1037">
  <Title>The Role of Lexico-Semantic Feedback in Open-Domain Textual Question-Answering</Title>
  <Section position="3" start_page="0" end_page="0" type="metho">
    <SectionTitle>
2 Related work
</SectionTitle>
    <Paragraph position="0"> Mechanisms for open-domain textual Q&amp;A were not discovered in the vacuum. The 90s witnessed a constant improvement of IR systems, determined by the availability of large collections of texts and the TREC evaluations. In parallel, Information Extraction (IE) techniques were developed under the TIPSTER Message Understanding Conference (MUC) competitions. Typically, IE systems identify information of interest in a text and map it to a predefined, target representation, known as template. Although simple combinations of IR and IE techniques are not practical solutions for open-domain textual Q&amp;A because IE systems are based on domain-specific knowledge, their contribution to current open-domain Q&amp;A methods is significant. For example, state-of-the-art Named Entity (NE) recognizers developed for IE systems were readily available to be incorporated in Q&amp;A systems and helped recognize names of people, organizations, locations or dates.</Paragraph>
    <Paragraph position="1"> Assuming that it is very likely that the answer is a named entity, (Srihari and Li, 2000) describes a NE-supported Q&amp;A system that functions quite well when the expected answer type is one of the categories covered by the NE recognizer. Unfortunately this system is not fully autonomous, as it depends on IR results provided by external search engines. Answer extractions based on NE recognizers were also developed in the Q&amp;A presented in (Abney et al., 2000) (Radev et al., 2000) (Gaizauskas and Humphreys, 2000). As noted in (Voorhees and Tice, 2000), Q&amp;A systems that did not include NE recognizers performed poorly in the TREC evaluations, especially in the short answer category. Some Q&amp;A systems, like (Moldovan et al., 2000) relied both on NE recognizers and some empirical indicators.</Paragraph>
    <Paragraph position="2"> However, the answer does not always belong to a category covered by the NE recognizer. For such cases several approaches have been developed. The first one, presented in (Harabagiu et al., 2000), the answer type is derived from a large answer taxonomy. A different approach, based on statistical techniques was proposed in (Radev et al., 2000). (Cardie et al., 2000) presents a method of extracting answers as noun phrases in a novel way. Answer extraction based on grammatical information is also promoted by the system described in (Clarke et al., 2000).</Paragraph>
    <Paragraph position="3"> One of the few Q&amp;A systems that takes into account morphological, lexical and semantic alternations of terms is described in (Ferret et al., 2000). To our knowledge, none of the current open-domain Q&amp;A systems use any feed-back loops to generate lexico-semantic alternations. This paper shows that such feedback loops enhance significantly the performance of open-domain textual Q&amp;A systems.</Paragraph>
  </Section>
  <Section position="4" start_page="0" end_page="0" type="metho">
    <SectionTitle>
3 Textual Q&amp;A Feedback Loops
</SectionTitle>
    <Paragraph position="0"> Before initiating the search for the answer to a natural language question we take into account the fact that it is very likely that the same question or a very similar one has been posed to the system before, and thus those results can be used again. To find such cached questions, we measure the similarity to the previously processed questions and when a reformulation is identified, the system returns the corresponding cached correct answer, as illustrated in Figure 1.</Paragraph>
    <Paragraph position="1"> When no reformulations are detected, the search for answers is based on the conjecture that the eventual answer is likely to be found in a text paragraph that (a) contains the most representative question concepts and (b) includes a textual concept of the same category as the expected answer. Since the current retrieval technology does not model semantic knowledge, we break down this search into a boolean retrieval, based on some question keywords and a filtering mechanism, that retains only those passages containing the expected answer type. Both the question key-words and the expected answer type are identified by using the dependencies derived from the question parse.</Paragraph>
    <Paragraph position="2"> By implementing our own version of the publicly available Collins parser (Collins, 1996), we also learned a dependency model that enables the mapping of parse trees into sets of binary relations between the head-word of each constituent and its sibling-words. For example, the parse tree of TREC-9 question Q210: &amp;quot;How many dogs pull a sled in the Iditarod ?&amp;quot; is:  dogs pull a sled in the For each possible constituent in a parse tree, rules first described in (Magerman, 1995) and (Jelinek et al., 1994) identify the head-child and propagate the head-word to its parent. For the parse of question Q210 the propagation is:</Paragraph>
    <Paragraph position="4"> pull a sled in the Iditarod</Paragraph>
    <Paragraph position="6"> When the propagation is over, head-modifier relations are extracted, generating the following dependency structure, called question semantic form in (Harabagiu et al., 2000).</Paragraph>
    <Paragraph position="7"> dogs IditarodCOUNT pull sled In the structure above, COUNT represents the expected answer type, replacing the question stem &amp;quot;how many&amp;quot;. Few question stems are unambiguous (e.g. who, when). If the question stem is ambiguous, the expected answer type is determined by the concept from the question semantic form that modifies the stem. This concept is searched in an ANSWER TAXONOMY comprising several tops linked to a significant number of WordNet noun and verb hierarchies. Each top represents one of the possible expected answer types implemented in our system (e.g. PERSON, PRODUCT, NUMERICAL VALUE, COUNT, LOCATION). We encoded a total of 38 possible answer types.</Paragraph>
    <Paragraph position="8"> In addition, the question keywords used for paragraph retrieval are also derived from the question semantic form. The question keywords are organized in an ordered list which first enumerates the named entities and the question quotations, then the concepts that triggered the recognition of the expected answer type followed by all adjuncts, in a left-to-right order, and finally the question head. The conjunction of the keywords represents the boolean query applied to the document index. (Moldovan et al., 2000) details the empirical methods used in our system for transforming a natural language question into an IR query.</Paragraph>
    <Paragraph position="9">  It is well known that one of the disadvantages of boolean retrieval is that it returns either too many or too few documents. However, for question answering, this is an advantage, exploited by the first feedback loop represented in Figure 1.</Paragraph>
    <Paragraph position="10"> Feedback loop 1 is triggered when the number of retrieved paragraphs is either smaller than a minimal value or larger than a maximal value determined beforehand for each answer type. Alternatively, when the number of paragraphs is within limits, those paragraphs that do not contain at least one concept of the same semantic category as the expected answer type are filtered out. The remaining paragraphs are parsed and their dependency structures, called answer semantic forms, are derived.</Paragraph>
    <Paragraph position="11"> Feedback loop 2 illustrated in Figure 1 is activated when the question semantic form and the answer semantic form cannot by unified. The unification involves three steps: a25 Step 1: The recognition of the expected answer type. The first step marks all possible concepts that are answer candidates. For example, in the case of TREC -9 question Q243: &amp;quot;Where did the ukulele originate ?&amp;quot;, the expected answer type is LOCATION. In the paragraph &amp;quot;the ukulele introduced from Portugal into the Hawaiian islands&amp;quot; contains two named entities of the category LOCATION and both are marked accordingly.</Paragraph>
    <Paragraph position="12"> a25 Step 2: The identification of the question concepts. The second step identifies the question words, their synonyms, morphological derivations or WordNet hypernyms in the answer semantic form.</Paragraph>
    <Paragraph position="13"> a25 Step 3: The assessment of the similarities of dependencies. In the third step, two classes of similar dependencies are considered, generating unifications of the question and answer semantic forms:a26 Class L2-1: there is a one-to-one mapping between the binary dependencies of the question and binary dependencies from the answer semantic form. Moreover, these dependencies largely cover the question semantic form2. An example  We find an entailment between producing, or making and selling goods, derived from Word-Net, since synset a27 make, produce, createa28 has the genus manufacture, defined in the gloss of its homomorphic nominalization as &amp;quot;for sale&amp;quot;. Therefore the semantic form of question Q261 and its illustrated answer are similar.a26 Class L2-2: Either the question semantic form or the answer semantic form contain new con2Some modifiers might be missing from the answer. cepts, that impose a bridging inference. The knowledge used for inference is of lexical nature and is later employed for abductions that justify the correctness of the answer. For example:  Nouns head and government are constituents of a possible paraphrase of president, i.e. &amp;quot;head of government&amp;quot;. However, only world knowledge can justify the answer, since there are countries where the prime minister is the head of government. Presupposing this inference, the semantic form of the question and answer are similar.</Paragraph>
    <Paragraph position="14"> Feedback loop 3 from Figure 1 brings forward additional semantic information. Two classes of similar dependencies are considered for the abduction of answers, performed in a manner similar to the justifications described in (Harabagiu et al., 2000).a26 Class L3-1: is characterized by the need for contextual information, brought forward by reference resolution. In the following example, a chain of coreference links Bill Gates and Microsoft founder in the candidate answer:  Class L3-2: Paraphrases and additional information produce significant differences between the question semantic form and the answer semantic form. However, semantic information contributes to the normalization of the answer dependencies until they can be unified with the question dependencies. For example, if (a) a volcano IS-A mountain; (b) lava IS-PART of volcano, and moreover it is a part coming from the inside; and (c) fragments of lava have all the properties of lava, the following question semantic form and answer semantic form can be unified:  The semantic information and the world knowledge needed for the above unifications are available from WordNet (Miller, 1995). Moreover, this knowledge can be translated in axiomatic form and used for abductive proofs. Each of the feedback loops provide the retrieval engine with new alternations of the question keywords. Feedback loop 2 considers morphological and lexical alternations whereas Feedback loop 3 uses semantic alternations. The method of generating the alternations is detailed in Section 4.</Paragraph>
  </Section>
  <Section position="5" start_page="0" end_page="0" type="metho">
    <SectionTitle>
4 Keyword Alternations
</SectionTitle>
    <Paragraph position="0"> To enhance the chance of finding the answer to a question, each feedback loop provides with a different set of keyword alternations. Such alternations can be classified according to the linguistic knowledge they are based upon: 1.Morphological Alternations. When lexical alternations are necessary because no answer was found yet, the first keyword that is altered is determined by the question word that either prompted the expected answer type or is in the same semantic class with the expected answer type. For example, in the case of question Q209: &amp;quot;Who invented the paper clip ?&amp;quot;, the expected answer type is PERSON and so is the subject of the verb invented , lexicalized as the nominalization inventor. Moreover, since our retrieval mechanism does not stem keywords, all the inflections of the verb are also considered.</Paragraph>
    <Paragraph position="1"> Therefore, the initial query is expanded into:  QUERY(Q209):a29paper AND clip AND (invented OR inventor OR invent OR invents)a30 2. Lexical Alternations. WordNet encodes a  wealth of semantic information that is easily mined. Seven types of semantic relations span concepts, enabling the retrieval of synonyms and other semantically related terms. Such alternations improve the recall of the answer paragraphs. For example, in the case of question Q221: &amp;quot;Who killed Martin Luther King ?&amp;quot;, by considering the synonym of killer, the noun assassin, the Q&amp;A system retrieved paragraphs with the correct answer. Similarly, for the question Q206: &amp;quot;How far is the moon ?&amp;quot;, since the adverb far is encoded in WordNet as being an attribute of distance, by adding this noun to the retrieval keywords, a correct answer is found.</Paragraph>
    <Paragraph position="2">  3. Semantic Alternations and Paraphrases. We  define as semantic alternations of a keyword those words or collocations from WordNet that (a) are not members of any WordNet synsets containing the original keyword; and (b) have a chain of WordNet relations or bigram relations that connect it to the original keyword. These relations can be translated in axiomatic form and thus participate to the abductive backchaining from the answer to the question - to justify the answer. For example semantic alternations involving only WordNet relations were used in the case of question Q258: &amp;quot;Where do lobsters like to live ?&amp;quot;. Since in WordNet the verb prefer has verb like as a hypernym, and moreover, its glossed definition is liking better, the query becomes: QUERY(Q258):a29lobsters AND (like OR prefer) AND live a30 Sometimes multiple keywords are replaced by a semantic alternation. Sometimes these alternations are similar to the relations between multiterm paraphrases and single terms, other time they simply are semantically related terms. In the case of question Q210: &amp;quot;How many dogs pull a sled in the Iditarod ?&amp;quot;, since the definition of Word-Net sense 2 of noun harness contains the bigram &amp;quot;pull cart&amp;quot; and both sled and cart are forms of vehicles, the alternation of the pair of keywords a31 pull, slide a32 is rendered by harness. Only when this feedback is received, the paragraph containing the correct answer is retrieved.</Paragraph>
    <Paragraph position="3"> To decide which keywords should be expanded and what form of alternations should be used we rely on a set of heuristics which complement the heuristics that select the question keywords and generate the queries (as described in (Moldovan et al., 2000)): Heuristic 1: Whenever the first feedback loop requires the addition of the main verb of the question as a query keyword, generate all verb conjugations as well as its nominalizations.</Paragraph>
    <Paragraph position="4"> Heuristic 2: Whenever the second feedback loop requires lexical alternations, collect from Word-Net all the synset elements of the direct hypernyms and direct hyponyms of verbs and nominalizations that are used in the query. If multiple verbs are used, expand them in a left-to-right order. null Heuristic 3: Whenever the third feedback loop imposes semantic alternations expressed as paraphrases, if a verb and its direct object from the question are selected as query keywords, search for other verb-object pairs semantically related to the query pair. When new pairs are located in the glosses of a synset a33 , expand the query verb-object pair with all the elements from a33 . Another set of possible alternations, defined by the existence of lexical relations between pairs of words from different question are used to detect question reformulations. The advantage of these different forms of alternations is that they enable the resolution of similar questions through answer caching instead of normal Q&amp;A processing. null</Paragraph>
  </Section>
  <Section position="6" start_page="0" end_page="0" type="metho">
    <SectionTitle>
5 Question Reformulations
</SectionTitle>
    <Paragraph position="0"> In TREC-9 243 questions were reformulations of 54 inquiries, thus asking for the same answer. The reformulation classes contained variable number of questions, ranging from two to eight questions.</Paragraph>
    <Paragraph position="1"> Two examples of reformulation classes are listed in Table 1. To classify questions in reformulation groups, we used the algorithm:  In Figure 2 we represent the similarity matrix for six questions that were successively posed to the answer engine. Since question reformulations are transitive relations, if at a step a39 questions a40a42a41 and a40a44a43 are found similar and a40 a41 already belongs to a45 , a reformulation class previously discovered (i.e. a group of at least two similar questions), then question a40 a43 is also included in a45 . Figure 2 illustrates the transitive closures for reformulations at each of the five steps from the succession of six questions. To be noted that at step 4 no new similarities were found , thus a40a47a46 is not found similar to a40a42a48 at this step. However, at step 5, since  The algorithm that measures the similarity between two questions is: Algorithm Similarity(Q, Q') Input: a pair of question represented as two word strings: Q: a52a54a53a7a52a56a55a56a57a58a57a58a57a59a52a14a60 and Q': a52a56a61a53 a52a56a61a55 a57a62a57a58a57a21a52a56a61a60 a57a58a57a62a57a59a52a14a63  if a52 a81 and a52a20a61a84 are content words with a66a77a67a70a69 a81a87a86 a66a77a67a70a69a73a61a84 and Lexical relationa79a80a52 a81 a82a83a52 a61a84 a85 holds then increase nr matches  4. Relax the Lexical relation and goto step 3; 5. If (nr matchesa64 number of content words a88a89a66 a85  then Q and Q' are similar The Lexical relation between a pair of content words is initially considered to be a string identity. In later loops starting at step 3 one of the following three possible relaxations of Lexical relation are allowed: (a) common morphological root (e.g. owner and owns, from question Q742: &amp;quot;Who is the owner of CNN ?&amp;quot; and question Q417: &amp;quot;Who owns CNN ?&amp;quot; respectively); (b) WordNet synonyms (e.g. gestation and pregnancy from question Q763: &amp;quot;How long is human gestation ?&amp;quot; and question Q765: &amp;quot;A normal human pregnancy lasts how many months ?&amp;quot;, respectively) or (c) WordNet hypernyms (e.g. the verbs erect and build from question Q814: &amp;quot;When was Berlin's Brandenburg gate erected ?&amp;quot; and question Q397: &amp;quot;When was the Brandenburg Gate in Berlin built ?&amp;quot; respectively).</Paragraph>
  </Section>
class="xml-element"></Paper>