<?xml version="1.0" standalone="yes"?>
<Paper uid="P06-1135">
  <Title>Improving QA Accuracy by Question Inversion</Title>
  <Section position="10" start_page="1078" end_page="1079" type="concl">
    <SectionTitle>
6 Summary
</SectionTitle>
    <Paragraph position="0"> We have extended earlier Constraints-based work through the method of question inversion. The approach uses our QA system recursively, by taking candidate answers and attempts to validate them through asking the inverted questions. The outcome  is a re-ranking of the candidate answers, with the possible insertion of nil (no answer in corpus) as the top answer.</Paragraph>
    <Paragraph position="1"> While we believe the approach is general, and can work on any question and arbitrary candidate lists, due to training limitations we focused on two restricted evaluations. In the first we used a fixed question type, and showed that the error rate was reduced by 36% and 30% on two very different corpora. In the second evaluation we focused on questions whose direct answers were correct in the second position. 43% of these questions were subsequently judged correct, at a cost of only 3.7% of originally correct questions. While in the future we would like to extend the Constraints process to the entire answer candidate list, we have shown that applying it only to the top two can be beneficial as long as the second-place answers are at least a tenth as numerous as first-place answers. We also showed that the application of Constraints can improve the system's confidence in its answers.</Paragraph>
    <Paragraph position="2"> We have identified several areas where improvement to our system would make the Constraints process more effective, thus getting a double benefit. In particular we feel that much more attention should be paid to the problem of determining if two entities are the same (or &amp;quot;close enough&amp;quot;).</Paragraph>
  </Section>
class="xml-element"></Paper>