<?xml version="1.0" standalone="yes"?>
<Paper uid="W03-1209">
  <Title>Statistical QA - Classifier vs. Re-ranker: What's the difference?</Title>
  <Section position="9" start_page="21" end_page="21" type="concl">
    <SectionTitle>
6 Conclusion
</SectionTitle>
    <Paragraph position="0"> The re-ranker system is very robust in handling large amounts of data and still produces reasonable results. There is no need for a major pre-processing step (for eliminating undesirable incorrect answers from the training) or the post-processing step (for selecting the most promising answer.) We also consider it significant that a QA system with just 4 features (viz. Frequency, Expected Answer Type, Question word absent, and ITF word match) is a good baseline system and performs better than the median performance of all the QA systems in the TREC 2002 evaluations5.</Paragraph>
    <Paragraph position="1"> Ittycheriah (2001), and Ittycheriah and Roukos (2002) have shown good results by using a range of features for Maximum Entropy QA systems. Also, the results indicate that there is scope for research in IR for QA systems. The QA system has an upper ceiling on performance due to the quality of the IR system. The QA community has yet to address these problems in a principled way, and the IR details of most of the system are hidden behind the complicated system architecture.</Paragraph>
    <Paragraph position="2"> The re-ranking model basically changes the objective function for training and the system is directly optimized on the evaluation function criteria (though still using Maximum Likelihood training). Also this approach seems to be very robust to noisy training data and is highly scalable. null Acknowledgements.</Paragraph>
    <Paragraph position="3"> This work was supported by the Advance Research and Development Activity (ARDA)'s Advanced Question Answering for Intelligence (AQUAINT) Program under contract number 5 However, since the IR system used here was from the Web, our results are not directly comparable with the TREC systems.</Paragraph>
    <Paragraph position="4"> MDA908-02-C-007. The authors wish to express particular gratitude to Dr. Abraham Ittycheriah, both for his supervision and education of the first author during his summer visit to IBM TJ Watson Research Center in 2002 and for his thoughtful comments on this paper, which was inspired by his work.</Paragraph>
  </Section>
class="xml-element"></Paper>