<?xml version="1.0" standalone="yes"?>
<Paper uid="A97-1026">
  <Title>An Automatic Scoring System For Advanced Placement Biology Essays</Title>
  <Section position="2" start_page="0" end_page="174" type="intro">
    <SectionTitle>
INTRODUCTION
</SectionTitle>
    <Paragraph position="0"> To replace the conventional multiple questions on standardized examinations, choice ~Test items in this paper are copyrighted by Educational Testing Service (ETS). No further reproduction is permitted without written permission of ETS.</Paragraph>
    <Paragraph position="1">  Educational Testing Service (ETS) is currently developing computer-based scoring tools for automatic scoring of natural language constructed-responses - responses that are written, such as a short-answer or an essay. The purpose of this work is to develop computer-based methods for scoring so that computer-administered natural language constructed-response items can be used on standardized tests and scored efficiently with regard to time and cost.</Paragraph>
    <Paragraph position="2"> Until recently, ETS's automated scoring efforts were primarily devoted to the development of computer programs used to score short-answer constructed-responses of up to 15 words (Burstein and Kaplan, 1995 and Burstein et al., 1996). In this study a classification of Excellent or Poor was automatically assigned to an AP Biology essay. Our initial goal in this study was to develop a prototype scoring system that could reliably assign a classification of Excellent to a set of AP Biology essays. For the evaluation of the scoring method, a small sample of Poor essays were also scored to compare the results. 2 Human rater scoring of AP Biology essays is based on a highly constrained scoring key, called a rubric, that specifies the criteria human raters use to assign scores to essays. Accordingly, for the test question studied here, the criteria for point</Paragraph>
  </Section>
class="xml-element"></Paper>