<?xml version="1.0" standalone="yes"?>
<Paper uid="P06-2073">
  <Title>dialogue models[?]</Title>
  <Section position="3" start_page="0" end_page="563" type="intro">
    <SectionTitle>
1 Introduction
</SectionTitle>
    <Paragraph position="0"> In the Natural Language Processing (NLP) field, one of the most challenging applications is dialogue systems (Kuppevelt and Smith, 2003). A dialogue system is usually defined as a computer system that can interact with a human being through dialogue in order to complete a specific task (e.g., ticket reservation, timetable consultation, bank operations,...) (Aust et al., 1995; Hardy et al., 2002). Most dialogue system have a characteristic behaviour with respect to dialogue [?] Work partially supported by the Spanish project TIC2003-08681-C02-02 and by Spanish Ministry of Culture under FPI grants.</Paragraph>
    <Paragraph position="1"> management, which is known as dialogue strategy. It defines what the dialogue system must do at each point of the dialogue.</Paragraph>
    <Paragraph position="2"> Most of these strategies are rule-based, i.e., the dialogue strategy is defined by rules that are usually defined by a human expert (Gorin et al., 1997; Hardy et al., 2003). This approach is usually difficult to adapt or extend to new domains where the dialogue structure could be completely different, and it requires the definition of new rules.</Paragraph>
    <Paragraph position="3"> Similar to other NLP problems (like speech recognition and understanding, or statistical machine translation), an alternative data-based approachhasbeendevelopedinthelastdecade(Stol- null cke et al., 2000; Young, 2000). This approach relies on statistical models that can be automatically estimated from annotated data, which in this case, are dialogues from the task.</Paragraph>
    <Paragraph position="4"> Statistical modelling learns the appropriate parameters of the models from the annotated dialogues. As a simplification, it could be considered that each label is associated to a situation in the dialogue, and the models learn how to identify and react to the different situations by estimating the associations between the labels and the dialogue events (words, the speaker, previous turns, etc.).</Paragraph>
    <Paragraph position="5"> An appropriate annotation scheme should be defined to capture the elements that are really important for the dialogue, eliminating the information that is irrelevant to the dialogue process. Several annotation schemes have been proposed in the last few years (Core and Allen, 1997; Dybkjaer and Bernsen, 2000).</Paragraph>
    <Paragraph position="6"> One of the most popular annotation schemes at the dialogue level is based on Dialogue Acts (DA).</Paragraph>
    <Paragraph position="7"> A DA is a label that defines the function of the annotated utterance with respect to the dialogue process. In other words, every turn in the dialogue  is supposed to be composed of one or more utterances. In this context, from the dialogue management viewpoint an utterance is a relevant sub-sequence . Several DA annotation schemes have been proposed in recent years (DAMSL (Core and Allen, 1997), VerbMobil (Alexandersson et al., 1998), Dihana (Alc'acer et al., 2005)).</Paragraph>
    <Paragraph position="8"> In all these studies, it is necessary to annotate a large amount of dialogues to estimate the parameters of the statistical models. Manual annotation is the usual solution, although is very time-consuming and there is a tendency for error (the annotation instructions are not usually easy to interpret and apply, and human annotators can commit errors) (Jurafsky et al., 1997).</Paragraph>
    <Paragraph position="9"> Therefore, the possibility of applying statistical models to the annotation problem is really interesting. Moreover, it gives the possibility of evaluating the statistical models. The evaluation of the performance of dialogue strategies models is a difficult task. Although many proposals have been made (Walker et al., 1997; Fraser, 1997; Stolcke et al., 2000), there is no real agreement in the NLP community about the evaluation technique to apply. null Our main aim is the evaluation of strategy models, which provide the reaction of the system given a user input and a dialogue history. Using these models as annotation models gives us a possible evaluation: the correct recognition of the labels implies the correct recognition of the dialogue situation; consequently this information can help the system to react appropriately. Many recent works have attempted this approach (Stolcke et al., 2000; Webb et al., 2005).</Paragraph>
    <Paragraph position="10"> However, many of these works are based on the hypothesis of the availability of the segmentation into utterances of the turns of the dialogue. This is an important drawback in order to evaluate these models as strategy models, where segmentation is usually not available. Other works rely on a decoupled scheme of segmentation and DA classification (Ang et al., 2005).</Paragraph>
    <Paragraph position="11"> In this paper, we present a new statistical model that computes the segmentation and the annotation of the turns at the same time, using a statistical framework that is simpler than the models that have been proposed to solve both problems at the same time (Warnke et al., 1997). The results demonstrate that segmentation accuracy is really important in obtaining an accurate annotation of the dialogue, and consequently in obtaining quality strategy models. Therefore, more accurate segmentation models are needed to perform this process efficiently.</Paragraph>
    <Paragraph position="12"> This paper is organised as follows: Section 2, presents the annotation models (for both the unsegmented and segmented versions); Section 3, describes the dialogue corpora used in the experiments; Section 4 establishes the experimental framework and presents a summary of the results; Section 5, presents our conclusions and future research directions.</Paragraph>
  </Section>
class="xml-element"></Paper>