<?xml version="1.0" standalone="yes"?>
<Paper uid="W04-2803">
  <Title>A Little Goes a Long Way: Quick Authoring of Semantic Knowledge Sources for Interpretation</Title>
  <Section position="3" start_page="0" end_page="0" type="intro">
    <SectionTitle>
2 Motivation
</SectionTitle>
    <Paragraph position="0"> While the technology presented in this paper is not specific to any particular application area, this work is motivated by a need within a growing community of researchers working on educational applications of Natural Language Processing to extract detailed information from student language input to be used for formulating specific feedback directed at the details of what the student has uttered. Such applications include tutorial dialogue systems (Zinn et al., 2002; Popescue et al., 2003) and writing coaches that perform detailed assessments of writing content (Ros'e et al., 2003; Wiemer-Hastings et al., 1998; Malatesta et al., 2002) as opposed to just grammar (Lonsdale and Strong-Krause, 2003), and provide detailed feedback rather than just letter grades (Burstein et al., 1998; Foltz et al., 1998). Because of the important role of language in the learning process (Chi et al., 2001), and because of the unique demands educational applications place on the technology, especially where detailed feedback based on student language input is offered to students, educational applications present interesting opportunities for this community.</Paragraph>
    <Paragraph position="1"> The area of automated essay grading has enjoyed a great deal of success at applying shallow language processing techniques to the problem of assigning general quality measures to student essays (Burstein et al., 1998; Foltz et al., 1998). The problem of providing reliable, detailed, content-based feedback to students is a more difficult problem, however, that involves identifying individual pieces of content (Christie, 2003), sometimes called &amp;quot;answer aspects&amp;quot; (Wiemer-Hastings et al., 1998). Previously, tutorial dialogue systems such as AUTO-TUTOR (Wiemer-Hastings et al., 1998) and Research Methods Tutor (Malatesta et al., 2002) have used LSA to perform an analysis of the correct answer aspects present in extended student explanations. While straightforward applications of bag of words approaches such as LSA have performed successfully on the content analysis task in domains such as Computer Literacy (Wiemer-Hastings et al., 1998), they have been demonstrated to perform poorly in causal domains such as research methods (Malatesta et al., 2002) and physics (Ros'e et al., 2003) because they base their predictions only on the words included in a text and not on the functional relationships between them. Key phrase spotting approaches such as (Christie, 2003) fall prey to the same problem. A hybrid rule learning approach to classification involving both statistical and symbolic features has been shown to perform better than LSA and Naive Bayes classification (McCallum and Nigam, 1998) for content analysis in the physics domain (Ros'e et al., 2003). Nevertheless, trained approaches such as this perform poorly on low-frequency classes and can be too coarse grained to provide enough information to the system for it to provide the kind of detailed feedback human tutors offer students (Lepper et al., 1993) unless an extensive hierarchy of classes that represent subtle differences in content is used (Popescue et al., 2003). Popescue et al. (2003) present impressive results at using a symbolic classification approach involving hand-written rules for performing a detailed assessment of student explanations in the Geometry domain.</Paragraph>
    <Paragraph position="2"> Rule based approaches have also shown promise in noneducational domains. For example, an approach to adapting the generic rule based MACE system for information extraction has achieved an F-measure of 82.2% at the ACE task (Maynard et al., 2002). Authoring tools for speeding up and simplifying the task of writing symbolic rules for assessing the content in student essays would make it more practical to take advantage of the benefits of rule based assessment approaches.</Paragraph>
  </Section>
class="xml-element"></Paper>