<?xml version="1.0" standalone="yes"?>
<Paper uid="N04-1012">
  <Title>Ensemble-based Active Learning for Parse Selection</Title>
  <Section position="7" start_page="50" end_page="50" type="concl">
    <SectionTitle>
7 Discussion
</SectionTitle>
    <Paragraph position="0"> We have shown that simple ensemble models can help both the underlying model and the AL method. Using a state-of-the-art parse selection model, we are able to achieve a a2a4a3a6a5 decrease in annotation costs compared against the highest performing single model trained using random sampling. This is one of the most substantial decreases in annotation cost reported in the literature. Our ensemble methods are very simple, and we expect that greater savings might follow when using more complex mode combination techniques such as boosting.</Paragraph>
    <Paragraph position="1"> We expect our parse selection-specific results to improve if we present only the top a17 most highly ranked parses to the annotator, rather than the full set of parses. Provided the true best parse is within the top a17 with sufficient regularity, this would reduce the number of discriminants which the human annotator needs to consider when compared to unaided uncertainty sampling.</Paragraph>
    <Paragraph position="2"> Another issue we will explore in future work is that for a scenario in which we label a data set from scratch, it is quite possible that we will not know how best to model the task we are labeling that data for. Thus, it is likely in such situations that we will be able to develop better evolved models only after the data is annotated and more has been learned about the task. It is then necessary to see whether improved models benefit from the examples selected using AL techniques with an earlier model more than they would have if random sampling had been used.</Paragraph>
  </Section>
class="xml-element"></Paper>