<?xml version="1.0" standalone="yes"?>
<Paper uid="W06-3503">
  <Title>Understanding Complex Natural Language Explanations in Tutorial Applications[?]</Title>
  <Section position="2" start_page="0" end_page="17" type="intro">
    <SectionTitle>
1 Introduction
</SectionTitle>
    <Paragraph position="0"> Most natural language tutorial applications have focused on coaching either problem solving or procedural knowledge (e.g. Steve (Johnson and Rickel, 1997), Circsim-tutor (Evens and Michael, 2006), Atlas (Ros'e et al., 2001), BEETLE (Zinn et al., 2002), SCoT (Peters et al., 2004), inter alia). When coaching problem solving, simple short answer analysis techniques are frequently sufficient because the primary goal is to lead a trainee step-by-step through problem solving. There is a narrow range of possible responses and the context of the previous dialogue and questions invite short answers. But when the instructional objectives shift and a tutorial system attempts to explore a student's chain of reasoning behind an answer or decision, deeper analysis techniques can begin to pay off. Having the student [?]This research was supported by ONR Grant No. N0001400-1-0600 and by NSF Grant No. 9720359.</Paragraph>
    <Paragraph position="1"> construct more on his own is important for learning perhaps in part because it reveals what the student does and does not understand (Chi et al., 2001).</Paragraph>
    <Paragraph position="2"> When the student is invited to provide a longer chain of reasoning, the explanations become multisentential. Compare the short explanation in Figure 1 to the longer ones in Figures 2 and 3. The explanation in Figure 2 is part of an actual initial student response and Figure 3 shows the explanation from the same student after a follow-up dialogue with the WHY2-ATLAS tutoring system.</Paragraph>
    <Paragraph position="3"> WHY2-ATLAS: Fine. Using this principle, what is the value of the horizontal component of the acceleration of the egg? Please explain your reasoning.</Paragraph>
    <Paragraph position="4"> Student: zero because there is no horizontal force acting on the egg [3 propositions expressed] Figure 1: Eliciting a one sentence explanation from a student.</Paragraph>
    <Paragraph position="5"> WHY2-ATLAS: Suppose a man is in an elevator that is falling without anything touching it (ignore the air, too). He holds his keys motionless right in front of his face and then just releases his grip on them. What will happen to them? Explain.</Paragraph>
    <Paragraph position="6"> Student: [omitted 15 correct propositions]... Yet the gravitational pull on the man and the elevator is greater because they are of a greater weight and therefore they will fall faster then the keys. I believe that the keys will float up to the cieling as the elevator continues falling.</Paragraph>
    <Paragraph position="7">  The only previous tutoring system that has attempted to address longer explanations is AUTOTU-TOR (Graesser et al., 2004). It uses a latent semantic  [omitted 16 correct propositions]... Since &lt;Net force = mass * acceleration&gt; and &lt;F= mass*g&gt; therefore &lt;mass*acceleration= mass*g&gt; and acceleration and gravitational force end up being equal. So mass does not effect anything in this problem and the acceleration of both the keys and the man are the same. [omitted 46 correct proposi- null dent in Figure 2 after some interaction with WHY2ATLAS. null analysis (LSA) approach where the structure of sentences is not considered. Thus the degree to which details of the explanation are understood is limited.</Paragraph>
    <Paragraph position="8"> As can be seen from the examples, a student's explanation about a formal domain such as qualitative physics may involve a number of phenomena: algebraic formulas, NL renderings of formulas, various degrees of formality, and conveying the logical structure of an argument (Makatchev et al., 2005).</Paragraph>
    <Paragraph position="9"> Tutoring goals involve eliciting correct statements of the appropriate degree of formality and their justifications to address possible gaps and errors in the explanation. To achieve these goals the NL understanding is required to answer the following questions: null Does the student explanation contain errors? If yes, what are the likely buggy assumptions that have led the student to these errors? What required statements have not been covered by the student? Does the explanation contain statements that are logically close to the required statements? These requirements imply that a logical structure needs to be imposed on the space of possible domain statements. Considering such a structure to be a model of the student's reasoning about the domain, the two requirements correspond to a solution of a model-based diagnosis problem (Forbus and de Kleer, 1993).</Paragraph>
    <Paragraph position="10"> How does one build such a model? A desire to make the process scalable and feasible necessitates an automated procedure. The difficulty is that this automated reasoner has to deal with the NL phenomena that are relevant for our application. In turn, this means that the knowledge representation (KR) would have to be able to express these phenomena (e.g. NL renderings of formulas, various degrees of formality). The reasoner has to account for common reasoning fallacies, have flexible consistency constraints and perform within the tight requirements of a real-time dialogue application.</Paragraph>
    <Paragraph position="11"> In this paper, we present a hybrid of symbolic and statistical approaches that attempts to robustly provide a model-based diagnosis of a student's explanation. In the next section, we provide a brief sketch of the KR used in WHY2-ATLAS. Section 3 describes our hybrid approach for analyzing student explanations while section 4 covers our most recent evaluations of the system and its explanation analysis components. Section 5 presents our conclusions along with future directions.</Paragraph>
  </Section>
class="xml-element"></Paper>