<?xml version="1.0" standalone="yes"?>
<Paper uid="P06-2041">
  <Title>Discriminative Classifiers for Deterministic Dependency Parsing</Title>
  <Section position="7" start_page="320" end_page="321" type="evalu">
    <SectionTitle>
5 Results and Discussion
</SectionTitle>
    <Paragraph position="0"> Table 2 shows the parsing accuracy for the combination of three languages (Swedish, English and Chinese), two learning methods (MBL and SVM) and five feature models (Ph1-Ph5), with algorithm parameters optimized as described in section 4.3.</Paragraph>
    <Paragraph position="1"> For each combination, we measure the attachment score (AS) and the exact match (EM). A significant improvement for one learning method over the other is marked by an asterisk (*).</Paragraph>
    <Paragraph position="2"> Independently of language and learning method, the most complex feature model Ph5 gives the highest accuracy across all metrics. Not surprisingly, the lowest accuracy is obtained with the simplest feature model Ph1. By and large, more complex feature models give higher accuracy, with one exception for Swedish and the feature models Ph3 and Ph4. It is significant in this context that the Swedish data set is the smallest of the three (about 20% of the Chinese data set and about 10% of the English one).</Paragraph>
    <Paragraph position="3"> If we compare MBL and SVM, we see that SVM outperforms MBL for the three most complex models Ph3, Ph4 and Ph5, both for English and Chinese. The results for Swedish are less clear, although the labeled accuracy for Ph3 and Ph5 are significantly better. For the Ph1 model there is no significant improvement using SVM. In fact, the small differences found in the ASU scores are to the advantage of MBL. By contrast, there is a large gap between MBL and SVM for the model Ph5 and the languages Chinese and English. For Swedish, the differences are much smaller (except for the EML score), which may be due to the smaller size of the Swedish data set in combination with the technique of dividing the training data for SVM (cf. section 3.2).</Paragraph>
    <Paragraph position="4"> Another important factor when comparing two learning methods is the efficiency in terms of time. Table 3 reports learning and parsing time for the three languages and the five feature models. The learning time correlates very well with the complexity of the feature model and MBL, being a lazy learning method, is much faster than SVM. For the unlexicalized feature models Ph1 and Ph2, the parsing time is also considerably lower for MBL, especially for the large data sets (English and Chinese). But as model complexity grows, especially with the addition of lexical features, SVM gradually gains an advantage over MBL with respect to parsing time. This is especially striking for Swedish,  where the training data set is considerably smaller than for the other languages.</Paragraph>
    <Paragraph position="5"> Compared to the state of the art in dependency parsing, the unlabeled attachment scores obtained for Swedish with model Ph5, for both MBL and SVM, are about 1 percentage point higher than the results reported for MBL by Nivre et al. (2004).</Paragraph>
    <Paragraph position="6"> For the English data, the result for SVM with model Ph5 is about 3 percentage points below the results obtained with the parser of Charniak (2000) and reported by Yamada and Matsumoto (2003).</Paragraph>
    <Paragraph position="7"> For Chinese, finally, the accuracy for SVM with model Ph5 is about one percentage point lower than the best reported results, achieved with a deterministic classifier-based approach using SVM and preprocessing to detect root nodes (Cheng et al., 2005a), although these results are not based on exactly the same dependency conversion and data split as ours.</Paragraph>
  </Section>
class="xml-element"></Paper>