<?xml version="1.0" standalone="yes"?>
<Paper uid="N06-1034">
  <Title>Modelling User Satisfaction and Student Learning in a Spoken Dialogue Tutoring System with Generic, Tutoring, and User Affect Parameters</Title>
  <Section position="8" start_page="269" end_page="270" type="concl">
    <SectionTitle>
5 Conclusions and Current Directions
</SectionTitle>
    <Paragraph position="0"> Prior work in the tutoring community has focused on correlations of single features with learning; our results suggest that PARADISE is an effective method of extending these analyses. For the dialogue community, our results suggest that as spoken dialogue systems move into new applications not optimized for user satisfaction, such as tutoring systems, other measures of performance may be more relevant, and generic user affect parameters may be useful.</Paragraph>
    <Paragraph position="1"> Our experiments used many of the same system-generic parameters as prior studies, and some of these parameters predicted user satisfaction both in our models and in prior studies' models (e.g., system words/turn (Walker et al., 2002)). Nonetheless, overall our user satisfaction models were not very powerful even for training, were sensitive to training data changes, showed little predictor overlap, and did not generalize well to test data. Our user satisfaction metric may not be ne-grained enough; in other PARADISE studies, users took a survey after every dialogue with the system. In addition, tutoring systems are not designed to maximize user satisfaction; their goal is to maximize student learning. Our student learning models were much more powerful and less sensitive to changes in training data. Our best models explained over 50% of the student learning variance for training and testing, and both student Correctness parameters and dialogue communication and ef ciency parameters were often useful predictors. User affect parameters further improved the predictive power of one student learning model for both training and testing.</Paragraph>
    <Paragraph position="2"> Once our user affect annotations are complete, we can further investigate their use to predict student learning and user satisfaction. Unlike our other parameters, these annotations are not currently available, although they can be predicted automatically (Litman and Forbes-Riley, 2004b), in our sys- null tem. However, as in (Batliner et al., 2003), our prior work suggests that linguistic features re ective of affective states can replace affect annotation (Forbes-Riley and Litman, 2005). In future work we will use such features in our prediction models. Finally, we are also annotating tutor and student dialogue acts and automating the tutor act annotations; when complete we can investigate their usefulness in our prediction models; dialogue acts have also been used in prior PARADISE applications (Mcurrency1oller, 2005a).</Paragraph>
  </Section>
class="xml-element"></Paper>