<?xml version="1.0" standalone="yes"?>
<Paper uid="W02-0207">
  <Title>Annotating Semantic Consistency of Speech Recognition Hypotheses</Title>
  <Section position="2" start_page="0" end_page="3" type="intro">
    <SectionTitle>
1 Introduction
</SectionTitle>
    <Paragraph position="0"> The complete understanding of naturally occurring discourse is still an unsolved task in computational linguistics. Several large research efforts are underway to build multi-domain and multimodal information systems, e.g. the DARPA Communicator Program  .</Paragraph>
    <Paragraph position="1"> Dialogue systems which deal with complex dialogues require the interaction of multiple knowledge sources, e.g. domain, discourse and user model (Flycht-Eriksson, 1999). Furthermore NLP systems have to adapt to different environments and applications. This can only be achieved if the system is able to determine how well a given speech recognition hypothesis (SRH) fits within the respective domain model and what domain should be considered by the system currently in focus. The purpose of this paper is to develop an annotation scheme for annotating a corpus of SRH with information on semantic consistency and domain specificity. We investigate</Paragraph>
    <Paragraph position="3"> the feasibility of an automatic solution by first looking at how reliably human annotators can solve the task.</Paragraph>
    <Paragraph position="4"> The structure of the paper is as follows: Section 2 gives an overview of the domain modeling component in the SmartKom system.</Paragraph>
    <Paragraph position="5"> In Section 3 we report on the data collection underlying our study. A description of the suggested annotation scheme is given in Section 4. Section 5 presents the results of an experiment in which the reliability of human annotations is investigated.</Paragraph>
  </Section>
class="xml-element"></Paper>