<?xml version="1.0" standalone="yes"?>
<Paper uid="A00-2037">
  <Title>Acknowledgments in Human-Computer Interaction</Title>
  <Section position="3" start_page="280" end_page="282" type="intro">
    <SectionTitle>
2 Experiment
</SectionTitle>
    <Paragraph position="0"> This study was designed as a pilot to our larger investigation into the effects of incorporating acknowledgement behavior in dialogue models for spoken-language interfaces. Before we attempted to compare interfaces with and without acknowledgement behavior, we wanted to understand whether people are willing to use this sort of metadialogue behavior when interacting with a computer. null</Paragraph>
    <Section position="1" start_page="280" end_page="280" type="sub_section">
      <SectionTitle>
2.1 Approach
</SectionTitle>
      <Paragraph position="0"> In this study we hypothesized that subjects will choose to use acknowledgments in human-computer interaction if they are given an interface that provides opportunities for and responds to acknowledgments.</Paragraph>
      <Paragraph position="1"> In designing the study, we assumed that it would not immediately occur to subjects that they could use acknowledgments to a computer. At the same time, we did not want to explicitly instruct or require subjects to use acknowledgment behavior, as that would tell us nothing about their preferences. We therefore decided against a comparison/ control-group experimental design for this initial study and instead focused on creating a situation in which subjects would have a reason to use acknowledgments, perhaps even gain an advantage from doing so, while still keeping the behavior optional.</Paragraph>
      <Paragraph position="2"> We decided to focus on a somewhat narrow use of acknowledgments. Conversants are especially likely to offer acknowledgments and repetitions when complex information is being presented, especially when the conversant is copying the information. While this is certainly explainable in terms of mutuality of understanding, this particular use of acknowledgment may be viewed from a more mechanical standpoint as regulating the pace at which information is presented. This insight suggested to us that a fruitful task for this study might be one in which the subject is asked to write down verbally-presented information, as when taking messages over the telephone.</Paragraph>
    </Section>
    <Section position="2" start_page="280" end_page="281" type="sub_section">
      <SectionTitle>
2.2 Task
</SectionTitle>
      <Paragraph position="0"> We selected the domain of telephone interface to email and designed a task in which subjects were asked to transcribe items of information from the messages. Writing is slow in comparison to speaking, so we anticipated that subjects would require a slower pace of information presentation when they were writing. The messages included information not asked for on the question list to simulate &amp;quot;uninteresting&amp;quot; material that the subject would want to move through at a faster pace. In this way we hoped to motivate subjects to try to control the pace at which information was presented.</Paragraph>
      <Paragraph position="1"> The email was presented in segments roughly corresponding to a long phrase. After each segment, the system paused to give the subject time to make notes. If the subject said nothing, the system would continue by presenting the next message segment. Subjects could accept--and perhaps make use of--this delay, or they could reduce it by acknowledging the contribution, e.g., &amp;quot;okay,&amp;quot; or by commanding the system to continue, e.g., &amp;quot;go on.&amp;quot; The system signalled the possibility of controlling the delay by prompting the subject &amp;quot;Are you ready to go on?&amp;quot; after the first pause. This prompting was repeated for every third pause in which the subject said nothing. In this way we hoped to suggest to the subjects that they could control the wait time if desired without explicitly telling them how to do SO.</Paragraph>
      <Paragraph position="2"> On the surface, there is no functional difference in system behavior between a subject's use of  a command to move the system onward (e.g., &amp;quot;go on&amp;quot; &amp;quot;next&amp;quot;, &amp;quot;continue&amp;quot;) and the use of an acknowledgment (&amp;quot;okay,&amp;quot; &amp;quot;uh-huh&amp;quot;, or a repetition). In either case, the system responds by presenting the next message segment, and in fact it eventually presents the next segment even if the subject says nothing at all. Thus, the design allows the subject to choose freely between accepting the system's pace (system initiative), or commanding the system to continue (user initiative), or acknowledging the presentations in a fashion more typical of mixed-initiative human conversation. In this way, we hoped to understand how the subject preferred to interact with the computer.</Paragraph>
    </Section>
    <Section position="3" start_page="281" end_page="281" type="sub_section">
      <SectionTitle>
2.3 Subjects
</SectionTitle>
      <Paragraph position="0"> Subjects were told that the study's purpose was to assess the understandability and usability of the interface, and that their task was to find the answers to a list of questions. They were given no instructions in the use of the program beyond the information that they could talk to it using normal, everyday speech.</Paragraph>
      <Paragraph position="1"> The 14 volunteers were native speakers of North American English, and most were staff at a research university. Ten were female, four were male. Ages ranged from 13 to 57. All used computers, typically office software and games, but none had significant programming experience. Each session lasted about 45 minutes total, and each subject was paid $10.00. One subject declined payment.</Paragraph>
    </Section>
    <Section position="4" start_page="281" end_page="282" type="sub_section">
      <SectionTitle>
2.4 Interface
</SectionTitle>
      <Paragraph position="0"> As mentioned earlier, one difficulty with recognizing acknowledgements in spoken-language interfaces is that the use of barge-in tends to defeat the purpose of acknowledgments when they occur in overlapped speech. We used a Wizard of Oz protocol as a simple way to allow the system to respond to such utterances and to provide robustness in handling repetitions.</Paragraph>
      <Paragraph position="1"> The wizard's interface was constructed using the Rapid Application Developer in the Center for Spoken Language Understanding Toolkit (Sutton, et al, 1998). A simple button panel allowed the wizard to select the appropriate response from the actions supported by the application. The application functionality was deliberately limited to suggest realistic abilities for a current spoken-language interface. Using messages pre-recorded in a synthesized voice, the wizard was able to direct the system to:  * Read a list of all messages.</Paragraph>
      <Paragraph position="2"> * Begin reading a particular message.</Paragraph>
      <Paragraph position="3"> * Read the next message segment.</Paragraph>
      <Paragraph position="4"> * Repeat the current message segment.</Paragraph>
      <Paragraph position="5"> * Repeat the previous message segment.</Paragraph>
      <Paragraph position="6"> * Ask the subject whether the program should continue reading the current message.</Paragraph>
      <Paragraph position="7"> * Ask the subject to what to do next.</Paragraph>
      <Paragraph position="8"> * End the program.</Paragraph>
      <Paragraph position="9"> * Play one of several error and help messages.  The texts of the email messages were presented in phrases of varying lengths, with each phrase followed by a pause of about five seconds. Preliminary tests showed that the combined response time of the wizard and the interface was between one and two seconds, and that pauses of less than five seconds were not obviously different from the normal pace of system response. Five seconds is a long response time, uncomfortably so for human-human conversation, so we hoped that this lengthy pause would encourage the subjects to take the initiative in controlling the pace of the interaction. null The messages were divided into segments by hand. The divisions were intended to simulate a phrase-level presentation, although some short phrases were combined to make the presentation less choppy. An example of one message and its division into phrases may be seen in Figure 1.</Paragraph>
      <Paragraph position="10"> Synthesized speech from the Festival speech synthesizer (Taylor, et al, 1998) was used throughout the interface. The message texts were presented in a synthesized male voice, while the control portions of the interface used a synthesized female voice. Default pronunciations were used except when the default was incorrect, e.g., &amp;quot;read&amp;quot; defaulted to the past-tense pronunciation in all contexts. Also, there was minor use of the SABLE markup language (Wouters, et al, 1999) to flatten the pitch range at the end of phrases in list items; the intent was to suggest the prosody of list continuation rather than the default sentence-final drop.  Message six is from Jo at teleport dot com, about, please stop by store on your way home. I'm going to be late getting home tonight, so would you please stop by the store on your way home? We need milk,  The subject's list of questions included &amp;quot;What items are you supposed to pick up at the store?&amp;quot; To improve the understandability, both voices were slowed slightly to 90% of the default speaking rate.</Paragraph>
    </Section>
    <Section position="5" start_page="282" end_page="282" type="sub_section">
      <SectionTitle>
2.5 Measures
</SectionTitle>
      <Paragraph position="0"> The central question to be answered is: will the subject use acknowledgments in interacting with the program? A subject can show one of several patterns of response: * The subject may make no attempt to control the pacing of the interface, instead allowing the interaction to proceed via time-outs.</Paragraph>
      <Paragraph position="1"> * The subject may use only commands to control the pacing.</Paragraph>
      <Paragraph position="2"> * The subject may use only acknowledgments to control the pacing.</Paragraph>
      <Paragraph position="3"> * The subject may use a mixture of commands and acknowledgments.</Paragraph>
      <Paragraph position="4"> The determination as to whether a particular utterance constituted an acknowledgment or a command was based primarily on word choice and dialogue context; this approach is consistent with definitions of this behavior, e.g., Chu-Carroll and Brown (1997). For example, &amp;quot;yes&amp;quot; in the context of a system inform (a segment of an email message) was considered an acknowledgment, but &amp;quot;yes&amp;quot; in the context of a system question was not. The words &amp;quot;okay,&amp;quot; &amp;quot;uh-huh,&amp;quot; and &amp;quot;yes&amp;quot; (immediately following an inform) were taken as evidence of acknowledgments, and phrases such as &amp;quot;go on,&amp;quot; &amp;quot;continue,&amp;quot; &amp;quot;next&amp;quot; following an inform were taken as evidence of commands. The interpretation was confirmed during the post-experiment interview by questioning the subjects about their word choice.</Paragraph>
    </Section>
    <Section position="6" start_page="282" end_page="282" type="sub_section">
      <SectionTitle>
2.6 Post-Experiment Interview
</SectionTitle>
      <Paragraph position="0"> A post-experiment interview was conducted to gather subject feedback and to answer subjects' questions. The experimenter took notes and thus could have introduced bias in the record of responsesdeg No tape recording was made.</Paragraph>
      <Paragraph position="1"> The subject was first invited to comment on the interface and the interaction in an open-ended fashion. When the subject had finished, the experimenter asked several specific questions to assess their understanding of the interface functionality.</Paragraph>
      <Paragraph position="2"> During this time, the experimenter reminded the subjects of the words that they had used most frequently to prompt the system to continue during pauses and asked the subjects why they had selected those words.</Paragraph>
      <Paragraph position="3"> Finally, the experimenter explained the true purpose and hypothesis of the experiment, verified that the subject was unaware that they had been interacting with a Wizard-of-Oz interface, and asked the subject to comment on the notion of using acknowledgments when interacting with a computer. The responses to this question, especially, must be assumed to be somewhat optimistic, as it is likely that at least some subjects would be reluctant to disagree with the experimenter.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>