<?xml version="1.0" standalone="yes"?>
<Paper uid="W05-1505">
  <Title>Corrective Modeling for Non-Projective Dependency Parsing</Title>
  <Section position="6" start_page="47" end_page="50" type="concl">
    <SectionTitle>
5 Empirical Results
</SectionTitle>
    <Paragraph position="0"> In this section we report results from experiments on the PDT Czech dataset. Approximately 1.9% of the words' dependencies are non-projective in version 1.0 of this corpus and these occur in 23.2% of the sentences (HajiVcov'a et al., 2004). We used the standard training, development, and evaluation datasets defined in the PDT documentation for all experiments. null  We use Zhang Lee's implementation of the  We have used PDT 1.0 (2002) data for the Charniak experiments and PDT 2.0 (2005) data for the Collins experiments. We use the most recent version of each parser; however we do not have a training program for the Charniak parser and have used the pretrained parser provided by Charniak; this was trained on the training section of the PDT 1.0. We train our model on the  Table 4 presents results on development data for the correction model with different feature sets. The features of the Simple model are the form (F), lemma (L), and morphological tag (M) for the each node, the parser-proposed governor node, and the candidate node; this model also contains the ParserGov feature. In the table's following rows, we show the results for the simple model augmented with feature sets of the categories described in Table 2. Table 3 provides a short description of each of the models. As we believe the Simple model provides the minimum information needed to perform this task, Collins trees via a 20-fold Jackknife training procedure.  Using held-out development data, we determined a Gaussian prior parameter setting of 4 worked best. The optimal number of training iterations was chosen on held-out data for each experiment. This was generally in the order of a couple hundred iterations of L-BFGS. The MaxEnt modeling implementation can be found at http://homepages.inf.ed.ac. uk/s0450736/maxent_toolkit.html.</Paragraph>
    <Paragraph position="1"> we experimented with the feature-classes as additions to it. The final row of Table 4 contains results for the model which includes all features from all other models.</Paragraph>
    <Paragraph position="2"> We define NonP Accuracy as the accuracy for the nodes which were non-projective in the original trees. Although both the Charniak and the Collins parser can never produce non-projective trees, the baseline NonP accuracy is greater than zero. This is due to the parser making mistakes in the tree such that the originally non-projective node's dependency is projective.</Paragraph>
    <Paragraph position="3"> Alternatively, we report the Non-Projective Precision and Recall for our experiment suite in Table 5. Here the numerator of the precision is the number of nodes that are non-projective in the correct tree and end up in a non-projective configuration; however, this new configuration may be based on incorrect dependencies. Recall is the obvious counterpart to precision. These values correspond to the NonP  parse trees.</Paragraph>
    <Paragraph position="4"> accuracy results reported in Table 4. From these tables, we see that the most effective features (when used in isolation) are the conjunctive MTag/Lemma,  for Collins' and Charniak's trees with and without the corrective model Finally, Table 6 shows the results of the full model run on the evaluation data for the Collins and Charniak parse trees. It appears that the Charniak parser fares better on the evaluation data than does the Collins parser. However, the corrective model is still successful at recovering non-projective structures. Overall, we see a significant improvement in the dependency accuracy.</Paragraph>
    <Paragraph position="5"> We have performed a review of the errors that the corrective process makes and observed that the model does a poor job dealing with punctuation.</Paragraph>
    <Paragraph position="6"> This is shown in Table 7 along with other types of nodes on which we performed well and poorly, respectively. Collins (1999) explicitly added features to his parser to improve punctuation dependency parsing accuracy. The PARSEVAL evaluation met-Top Five Good/Bad Repairs Well repaired child seisiaVzjen Well repaired false governor vvVsak li na o Well repaired real governor ajest'at ba , Poorly repaired child ,senaVze Poorly repaired false governor a,vVsak mus'ili Poorly repaired real governor root sklo , je - null made by our model on trees from the Charniak parser. root is the artificial root node of the PDT tree. For each node position (child, proposed parent, and correct parent), the top five words are reported (based on absolute count of occurrences). The particle 'se' occurs frequently explaining why it occurs in the top five good and top five bad repairs.</Paragraph>
    <Paragraph position="7">  model on Charniak and Collins trees.</Paragraph>
    <Paragraph position="8"> ric for constituency-based parsing explicitly ignores punctuation in determining the correct boundaries of constituents (Harrison et al., 1991) and so should the dependency evaluation. However, the reported results include punctuation for comparative purposes. Finally, we show in Table 8 a coarse analysis of the corrective performance of our model. We are repair- null ing more dependencies than we are corrupting.</Paragraph>
    <Section position="1" start_page="50" end_page="50" type="sub_section">
      <SectionTitle>
6Conclusion
</SectionTitle>
      <Paragraph position="0"> We have presented a Maximum Entropy-based corrective model for dependency parsing. The goal is to recover non-projective dependency structures that are lost when using state-of-the-art constituency-based parsers; we show that our technique recovers over 50% of these dependencies. Our algorithm provides a simple framework for corrective modeling of dependency trees, making no prior assumptions about the trees. However, in the current model, we focus on trees with local errors. Overall, our technique improves dependency parsing and provides the necessary mechanism to recover non-projective structures.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>