<?xml version="1.0" standalone="yes"?>
<Paper uid="P06-2085">
  <Title>Using Machine Learning to Explore Human Multimodal Clarification Strategies</Title>
  <Section position="9" start_page="664" end_page="665" type="concl">
    <SectionTitle>
7 Summary and Future Work
</SectionTitle>
    <Paragraph position="0"> We showed that humans use a context-dependent strategy for asking multimodal clarification requests by learning such a strategy from WOZ data.</Paragraph>
    <Paragraph position="1"> Only the two wizards with the lowest performance scores showed no significant variation across sessions, leading us to hypothesise that the better wizards converged on a context-dependent strategy.</Paragraph>
    <Paragraph position="2"> We were able to discover a runtime context based on which all wizards behaved uniformly, using feature discretisation methods and feature selection methods on dialogue context features. Based on these features we were able to predict how an 'average' wizard would behave in that context with an accuracy of 84.6% (wf-score of 85.3%, which is a 25.5% improvement over a one rule-based baseline). We explained the learned strategies and showed that they can be implemented in  rule-based dialogue systems based on domain independent features. We also showed that feature engineering is essential for achieving significant performance gains when using large feature spaces with the small data sets which are typical of dialogue WOZ studies. By interpreting the learnt strategies we found them to be sub-optimal. In current research, RL is applied to optimise strategies and has been shown to lead to dialogue strategies which are better than those present in the original data (Henderson et al., 2005). The next step towards a RL-based system is to add task-level and reward-level annotations to calculate reward functions, as discussed in (Rieser et al., 2005). We furthermore aim to learn more refined clarification strategies indicating the problem source and its severity.</Paragraph>
  </Section>
class="xml-element"></Paper>