<?xml version="1.0" standalone="yes"?>
<Paper uid="P06-2008">
  <Title>Sydney, July 2006. c(c)2006 Association for Computational Linguistics Towards Conversational QA: Automatic Identification of Problematic Situations and User Intent [?]</Title>
  <Section position="3" start_page="0" end_page="57" type="intro">
    <SectionTitle>
1 Introduction
</SectionTitle>
    <Paragraph position="0"> Interactive question answering (QA) has been identified as one of the important directions in QA research (Burger et al., 2001). One ultimate goal is to support intelligent conversation between a user and a QA system to better facilitate user information needs. However, except for a few systems that use dialog to address complex questions (Small et al., 2003; Harabagiu et al., 2005), the general dialog capabilities have been lacking in most ques[?] null This work was partially supported by IIS-0347548 from the National Science Foundation.</Paragraph>
    <Paragraph position="1"> tion answering systems. To move towards conversational QA, it is important to examine key issues relevant to conversational systems in the context of interactive question answering.</Paragraph>
    <Paragraph position="2"> This paper focuses on two issues related to conversational QA. The first issue is concerned with user intent. In conversational systems, understanding user intent is the key to the success of the interaction. In the context of interactive QA, one question is what type of user intent should be captured. Unlike most dialog systems where user intent can be characterized by dialog acts such as question, reply, and statement, in interactive QA, user inputs are already in the form of question. Then the problems become whether there are different types of intent behind these questions that should be handled differently by a QA system and how to automatically identify them.</Paragraph>
    <Paragraph position="3"> The second issue is concerned with problematic situations during interaction. In spoken dialog systems, many problematic situations could arise from insufficient speech recognition and language understanding performance. Recent work has shown that the capability to automatically identify problematic situations (e.g., speech recognition errors) can help control and adapt dialog strategies to improve performance (Litman and Pan, 2000). Similarly, QA systems also face challenges of technology limitation from language understanding and information retrieval. Thus one question is, in the context of interactive QA, how to characterize problematic situations and automatically identify them when they occur.</Paragraph>
    <Paragraph position="4"> In interactive QA, these two issues are intertwined. Questions formed by a user not only depend on his/her information goals, but are also influenced by the answers from the system. Problematic situations will impact user intent in the  follow-up questions, which will further influence system performance. Both the awareness of problematic situations and understanding of user intent will allow QA systems to adapt better strategies during interaction and move towards intelligent conversational QA.</Paragraph>
    <Paragraph position="5"> To address these two questions, we conducted a user study where users interacted with a controlled QA system to find information of interest. These controlled studies allowed us to focus on the interaction aspect rather than information retrieval or answer extraction aspects. Our studies indicate that in basic interactive QA where users always ask questions and the system always provides some kind of answers, there are different types of user intent that are tied to different kinds of system performance (e.g., problematic/error free situations). Once users are motivated to find specific information related to their information goals, the interaction context can provide useful cues for the system to automatically identify problematic situations and user intent.</Paragraph>
  </Section>
class="xml-element"></Paper>