<?xml version="1.0" standalone="yes"?>
<Paper uid="P03-2027">
  <Title>Dialog Navigator : A Spoken Dialog Q-A System based on Large Text Knowledge Base</Title>
  <Section position="3" start_page="0" end_page="1" type="metho">
    <SectionTitle>
2 Precise Text Retrieval
</SectionTitle>
    <Paragraph position="0"> It is critical for a Q-A system to retrieve relevant texts for a question precisely. In this section, we describe the score calculation method, giving large points to modifier-head relations between bunsetsu  based on the parse results of KNP (Kurohashi and Nagao, 1994), to improve precision of text retrieval. Our system also uses question types, product names, and synonymous expression dictionary as described in (Kiyota et al., 2002).</Paragraph>
    <Paragraph position="1"> First, scores of all sentences in each text are calculated as shown in Figure 2. Sentence score is the total points of matching keywords and modifier-head relations. We give 1 point to a matching of a keyword, and 2 points to a matching of a modifier-head relation (these parameters were set experimentally). Then sentence score is normalized by the maximum matching score (MMS) of both sentences as follows (the MMS is the sentence score with itself):  Finally, the sentence that has the largest score in each text is selected as the representative sentence of the text. Then, the score of the sentence is regarded as the score of the text.</Paragraph>
  </Section>
  <Section position="4" start_page="1" end_page="2" type="metho">
    <SectionTitle>
3 Dialog Strategy for Clarifying Questions
</SectionTitle>
    <Paragraph position="0"> In most cases, users' questions are vague. To cope with such vagueness, our system uses the following two methods: asking backs using dialog cards and extraction of summaries that makes difference between retrieved texts more clear (Figure 3).</Paragraph>
    <Section position="1" start_page="1" end_page="1" type="sub_section">
      <SectionTitle>
3.1 Dialog cards
</SectionTitle>
      <Paragraph position="0"> If a question is very vague, it matches many texts, so users have to pay their labor on finding a relevant one. Our system navigates users to the desired answer using dialog cards as shown in Figure 3.</Paragraph>
      <Paragraph position="1"> We made about three hundred of dialog cards to throw questions back to users. Figure 4 shows two dialog cards. BOUQBQ (User Question) is followed by a typical vague user question. If a user question matches it, the dialog manager asks the back question after BOSYSBQ, showing choices be- null Windows 95 retrieve Windows 95 wo kidou ji ni error ga hassei suru 'An error occurs while booting Windows 95' Windows 98 retrieve Windows 98 wo kidou ji ni error ga hassei suru 'An error occurs while booting Windows 98' Windows ME retrieve Windows ME wo kidou ji ni error ga hassei suru 'An error occurs while booting Windows ME'  followed by goto or retrieve. goto means that the system follow the another dialog cards if this choice is selected. retrieve means that the system retrieve texts using the query specified there.</Paragraph>
    </Section>
    <Section position="2" start_page="1" end_page="1" type="sub_section">
      <SectionTitle>
3.2 Description extraction from retrieved texts
</SectionTitle>
      <Paragraph position="0"> In most cases, the neighborhood of the part that matches the user question describes specific symptoms and conditions of the problem users encounter.</Paragraph>
      <Paragraph position="1"> Our system extracts such descriptions from the retrieved texts as the summaries of them. The algorithm is described in (Kiyota et al., 2002).</Paragraph>
      <Paragraph position="2"> 4 Dialog Strategy for Speech Input It is necessary for a spoken dialog system to determine which portions of the speech input should be confirmed. Moreover, criteria for judging whether it should make confirmation or not are needed, because too many confirmations make the dialog inefficient. Therefore, we introduce two criteria of confidence in recognition and significance for retrieval. Our system makes two types of asking backs for fixing recognition errors (Figure 1). First, Julius outputs C6-best candidates of speech recognition. Then, the system makes confirmation for significant parts based on confidence in recognition. After that, the system retrieves relevant texts in the text knowledge base using each candidate, and makes confirmation based on significance for retrieval.</Paragraph>
    </Section>
    <Section position="3" start_page="1" end_page="2" type="sub_section">
      <SectionTitle>
4.1 Confidence in recognition
</SectionTitle>
      <Paragraph position="0"> We define the confidence in recognition for each phrase in order to reject partial recognition errors. It is calculated based on word perplexity, which is often used in order to evaluate suitability of language models for test-set sentences. We adopt word perplexity because of the following reasons: incorrectly recognized parts are often unnatural in context, and words that are unnatural in context have high perplexity values.</Paragraph>
      <Paragraph position="1"> As Julius uses trigram as its language model, the word perplexity C8C8 is calculated as follows:  C8C8s are summed up in each bunsetsu (phrases).</Paragraph>
      <Paragraph position="2"> As a result, the system assigned the sum of C8C8s to each bunsetsu as the criterion for confidence in recognition.</Paragraph>
      <Paragraph position="3"> We preliminarily defined the set of product names as significant phrases  . If the sums of C8C8s for any significant phrases are beyond the threshold (now, we set it 50), the system makes confirmation for these phrases.</Paragraph>
    </Section>
    <Section position="4" start_page="2" end_page="2" type="sub_section">
      <SectionTitle>
4.2 Significance for retrieval
</SectionTitle>
      <Paragraph position="0"> The system calculates significance for retrieval using C6-best candidates of speech recognition. Because slight speech recognition errors are not harmful for retrieval results, we regard a difference that affects its retrieval result as significant. Namely, when the difference between retrieval results for each recognition candidate is large, we regard that the difference is significant.</Paragraph>
      <Paragraph position="1"> Significance for retrieval is defined as a rate of disagreement of five high-scored retrieved texts among C6 recognition candidates. For example, if there is a substituted part in two recognition candidates, and only one text is commonly retrieved out of five high-scored texts by both candidates, the significance for retrieval for the substituted part is 0.8 (BPBDA0 BDBPBH).</Paragraph>
      <Paragraph position="2"> The system makes confirmation which candidate should be used, if significance for retrieval is beyond the threshold (now, we set it 0.5).</Paragraph>
      <Paragraph position="3">  We are now developing a method to define the set of significant phrases semi-automatically.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>