<?xml version="1.0" standalone="yes"?>
<Paper uid="N06-1035">
  <Title>Comparing the Utility of State Features in Spoken Dialogue Using Reinforcement Learning</Title>
  <Section position="8" start_page="277" end_page="277" type="relat">
    <SectionTitle>
6 Related Work
</SectionTitle>
    <Paragraph position="0"> RL has been applied to improve dialogue systems in past work but very few approaches have looked at which features are important to include in the dialogue state. Paek and Chickering's (2005) work on testing the Markov Assumption for Dialogue Systems showed how the state space can be learned from data along with the policy. One result is that a state space can be constrained by only using features that are relevant to receiving a reward. Henderson et al.'s (2005) work focused on learning the best policy by using a combination of reinforcement and supervised learning techniques but also addressed state features by using linear function approximation to deal with large state spaces. Singh et al. (1999) and Frampton et al. (2005) both showed the effect of adding one discourse feature to the student state (dialogue length and user's last dialogue act, respectively) whereas in our work we compare the worth of multiple features. Although Williams et al.'s (2005b) work did not focus on choosing the best state features, they did show that in a noisy environment, Partially-Observable MDP's could be used to build a better model of what state the user is in, over traditional MDP and hand-crafted methods. One major difference between all this related work and ours is that usually the work is focused on how to best deal with ASR errors. Although this is also important in the tutoring domain, our work is novel because it focuses on more semantically-oriented questions.</Paragraph>
  </Section>
class="xml-element"></Paper>