<?xml version="1.0" standalone="yes"?>
<Paper uid="A00-2005">
  <Title>Bagging and Boosting a Treebank Parser</Title>
  <Section position="3" start_page="0" end_page="34" type="intro">
    <SectionTitle>
2 Bagging and Parsing
2.1 Background
</SectionTitle>
    <Paragraph position="0"> The work of Efron and Tibshirani (1993) enabled Breiman's refinement and application of their techniques for machine learning (Breiman, 1996). His technique is called bagging, short for &amp;quot;bootstrap aggregating&amp;quot;. In brief, bootstrap techniques and bagging in particular reduce the systematic biases many estimation techniques introduce by aggregating estimates made from randomly drawn representative resamplings of those datasets.</Paragraph>
    <Paragraph position="1"> Bagging attempts to find a set of classifiers which are consistent with the training data, different from each other, and distributed such that the aggregate sample distribution approaches the distribution of samples in the training set.</Paragraph>
    <Paragraph position="2"> Algorithm: Bagging Predictors (Breiman, 1996) (1) Given: training set = {(yi,x~),i E {1...m}} drawn from the set A of possible training sets where Yi is the label for example x~, classification induction algorithm q2 : A --* * with classification algorithm Ce * and C/: X--~Y.</Paragraph>
    <Paragraph position="3">  1. Create k bootstrap replicates of/: by sampling m items from E with replacement. Call them L1...Lk.</Paragraph>
    <Paragraph position="4"> 2. For each j e {1...k}, Let Cj = ~(PSj) be the classifier induced using Lj as the training set. 3. If Y is a discrete set, then for each x~ observed in the test set, yi = mode(C/j(xi)... Cj(x~)). y~ is the value predicted by the most predictors, the majority vote.</Paragraph>
    <Section position="1" start_page="0" end_page="34" type="sub_section">
      <SectionTitle>
2.2 Bagging for Parsing
</SectionTitle>
      <Paragraph position="0"> An algorithm that applies the technique of bagging to parsing is given in Algorithm 2. Previous work on combining independent parsers is leveraged to produce the combined parser. The rest of the algorithm is a straightforward transformation of bagging for classifiers. Exploratory work in this vein was described by HajiC et al. (1999).</Paragraph>
      <Paragraph position="1"> Algorithm: Bagging A Parser (2) Given: A corpus (again as a function)C:SxT ~ N, S is the set of possible sentences, and T is the set of trees, with size m = \[C\] = ~s,t C(s, t) and parser  induction algorithm g.</Paragraph>
      <Paragraph position="2"> 1. Draw k bootstrap replicates C1 ... Ck of C each containing m samples of (s,t) pairs randomly  picked from the domain of C according to the distribution D(s,t) = C(s,t)/\]C\]. Each bootstrap replicate is a bag of samples, where each sample in a bag is drawn randomly with replace- null ment from the bag corresponding to C.</Paragraph>
      <Paragraph position="3"> 2. Create parser f~ = g(Ci) for each i.</Paragraph>
      <Paragraph position="4"> 3. Given a novel sentence 8test E Ctest, combine the collection of hypotheses ti = fi(Stest) using the unweighted constituent voting scheme</Paragraph>
    </Section>
    <Section position="2" start_page="34" end_page="34" type="sub_section">
      <SectionTitle>
of Henderson and Brill (1999).
2.3 Experiment
</SectionTitle>
      <Paragraph position="0"> The training set for these experiments was sections 01-21 of the Penn Treebank (Marcus et al., 1993).</Paragraph>
      <Paragraph position="1"> The test set was section 23. The parser induction algorithm used in all of the experiments in this paper was a distribution of Collins's model 2 parser (Collins, 1997). All comparisons made below refer to results we obtained using Collins's parser.</Paragraph>
      <Paragraph position="2"> The results for bagging are shown in Figure 2 and  training set F-measure ~, test set F-measure, percent perfectly parsed sentences in training set, and percent perfectly parsed sentences in test set. An ensemble of bags was produced one bag at a time. In the table, the Initial row shows the performance achieved when the ensemble contained only one bag, Final(X) shows the performance when the ensemble contained X bags, BestF gives the performance of the ensemble size that gave the best F-measure score. TrainBestF and TestBestF give the test set performance for the ensemble size that performed the best on the training and test sets, respectively. On the training set all of the accuracy measures are improved over the original parser, and on the test set there is clear improvement in precision and recall. The improvement on exact sentence accuracy for the test set is significant, but only marginally so. The overall gain achieved on the test set by bagging was 0.8 units of F-measure, but because the entire corpus is not used in each bag the initial performance is approximately 0.2 units below the best previously reported result. The net gain using this technique is 0.6 units of F-measure.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>