<?xml version="1.0" standalone="yes"?>
<Paper uid="W04-3202">
  <Title>Active Learning and the Total Cost of Annotation</Title>
  <Section position="1" start_page="0" end_page="0" type="abstr">
    <SectionTitle>
Abstract
</SectionTitle>
    <Paragraph position="0"> Active learning (AL) promises to reduce the cost of annotating labeled datasets for trainable human language technologies.</Paragraph>
    <Paragraph position="1"> Contrary to expectations, when creating labeled training material for HPSG parse selection and later reusing it with other models, gains from AL may be negligible or even negative. This has serious implications for using AL, showing that additional cost-saving strategies may need to be adopted. We explore one such strategy: using a model during annotation to automate some of the decisions. Our best results show an 80% reduction in annotation cost compared with labeling randomly selected data with a single model.</Paragraph>
  </Section>
class="xml-element"></Paper>