<?xml version="1.0" standalone="yes"?>
<Paper uid="P06-1113">
  <Title>Question Answering with Lexical Chains Propagating Verb Arguments</Title>
  <Section position="7" start_page="901" end_page="902" type="evalu">
    <SectionTitle>
4 Experiments and Results
</SectionTitle>
    <Paragraph position="0"> The algorithm for propagating verb arguments was used to improve performance of an in-house Question Answering system (Moldovan et al., 2004).</Paragraph>
    <Paragraph position="1"> This improvement comes from a better matching between a question and the sentences containing the correct answer. Integration of this algorithm into the Question Answering system requires 3 steps: (1) creation of structures containing verb arguments for the questions and its possible answers, (2) derivation of lexical chains between the two structures and propagation of the arguments along lexical chains, (3) measuring the similarity between the propagated structures and the structures from the question and re-ranking of the candidate answers based on similarity scores. Structures containing predicate arguments are created for all the verbs in the question and all verbs in each possible answer. The QA system takes care of coreference resolution.</Paragraph>
    <Paragraph position="2"> Argument structures are created for verbs in both active and passive voice. If the verb is in passive voice, then its arguments are normalized to active voice. The subject phrase of the verb in passive voice represents its object and the noun phrase inside prepositional phrase with preposition &amp;quot;by&amp;quot; becomes its subject. Special attention is given to di-transitive verbs. If in passive voice, the sub-ject phrase can represent either the direct object or indirect object. The distinction is made in the following way: if the verb in passive voice has a direct object then the subject represents the indirect object (beneficiary), otherwise the subject represents direct object. All the other arguments are treated in the same way as in the active voice case.</Paragraph>
    <Paragraph position="3"> After the structures are created from a candidate answer and a question, lexical chains are created between their heads. Because lexical chains link two word senses, the heads need to be disambiguated. Before searching for lexical chains, the heads could be already partially disambiguated, because only a restricted number of senses of the head verb can have the VerbNet syntactic pattern matching the input text. An additional semantic disambiguation can take place before deriving lexical chains. The verbs from the answer and question can also be disambiguated by selecting the best lexical chain between them. This was the approach used in our experiment.</Paragraph>
    <Paragraph position="4"> The algorithm propagating verb arguments was tested on a set of 106 pairs of phrases with similar meaning for which argument structures could be built. These phrases were selected from pairs of questions and their correct answers from the  set of factoid questions in TREC 2004 and also from the pairs of scenarios and hypotheses from first edition of PASCAL RTE Challenge (Dagan et al., 2005). Table 6 shows algorithm performance.</Paragraph>
    <Paragraph position="5"> The columns in the table correspond to the following cases: a) how many cases the algorithm propagated all the arguments; b) how many cases the algorithm propagated one argument; c) home many cases the algorithm did not propagate any argument; using top 5, 20, 50 lexical chains.</Paragraph>
    <Paragraph position="6"> The purpose of the algorithm for propagating predicate arguments is to measure the similarity between the sentences for which the argument structures have been built. This similarity can be computed by comparing the target argument structure with the propagated argument structure. The similarity score is computed in the following way: ifa50 represents the number of arguments in a pattern, each argument matched is defined to have a contribution of a31a1a0a3a2a50a5a4 a31a1a6 , except for the subject that has a contribution if matched of 2/(N+1). The propagated pattern is compared with the target pattern and the score is computed by summing up the contributions of all matched arguments.</Paragraph>
    <Paragraph position="7"> The set of factoid questions in TREC 2004 has 230 questions. Lexical chains containing the restricted set of relations that propagate verb arguments were found for 33 questions, linking verbs in those questions to verbs in their correct answer. This is the maximum number of questions on which the algorithm for propagating syntactic constraints can have an impact without using other knowledge. The algorithm for propagating verb argument could be applied on 15 of these questions. Table 7 shows the improvement of the Question Answering system when the first 20 or 50 answers returned by factoid strategy are re-ranked according to similarity scores between argument structures. The performance of the question answering system was measured using Mean Reciprocal Rank (MRR).</Paragraph>
  </Section>
class="xml-element"></Paper>