<?xml version="1.0" standalone="yes"?>
<Paper uid="W03-0202">
  <Title>Learning to Identify Student Preconceptions from Texta0</Title>
  <Section position="7" start_page="0" end_page="0" type="concl">
    <SectionTitle>
6 Remarks
</SectionTitle>
    <Paragraph position="0"> Currently, the only processing of the student text done by the system is the removal of stopwords and stemming. It would be interesting to preprocess the text with part-of-speech tagging and syntactic and grammatical analysis, such as identification of passive or active voice or even full parsing. Because of the broad range of ways in which students express their ideas, the system may be severely hampered by limited exposure to syntactic variation. Traditional NLP analyses might allow for the creation of rule analogues. For example, a rule that matched &amp;quot;subtract the initial position from the final position&amp;quot; might be mapped to another rule that could match &amp;quot;take the final position and subtract the initial position.&amp;quot; The application of such methods might be complicated by the fact that student writing is often highly ungrammatical and short-answer responses may well be more so.</Paragraph>
    <Paragraph position="1"> Another way to improve the performance of automatic text analysis in assessing students is to take some care in constructing the problems presented to students to ease analysis of their answers. In the responses that were assigned to Pfinal-avg, students described various qualitative comparisons between the two lines. The lines were labeled as Object A and Object B on the graph. In their responses, students referred to them as &amp;quot;object A&amp;quot;, &amp;quot;line A&amp;quot;, &amp;quot;graph A&amp;quot; and just &amp;quot;A&amp;quot;. Since &amp;quot;A&amp;quot; is a common stopword, this effected our ability to learn rules for this preconception. We removed &amp;quot;A&amp;quot; from our stopword list, which allowed for different rules to be learned, but also allowed other rules to include &amp;quot;A&amp;quot; when it was being used as an indefinite article. The use of part-of-speech tagging may improve this situation, but so would changing the question to label the graph in a way that would be less confusing to the system.</Paragraph>
    <Paragraph position="2"> Key factors in the success or failure of experiments such as these are the variety of messages that must be mapped into a single category and degree to which usage of various words and patterns of words is consistent in implicating one category rather than another. Ultimately, the utility of techniques such as those we are studying may depend on the careful scoping of these categories and means to bias student writing towards particular styles or vocabularies. These techniques offer one approach to language analysis that lies between the purely syntactic and the thoroughly semantic ends of the spectrum. We are optimistic about their practical potential in the realm of educational assessment.</Paragraph>
  </Section>
class="xml-element"></Paper>