<?xml version="1.0" standalone="yes"?>
<Paper uid="P96-1024">
  <Title>Parsing Algorithms and Metrics</Title>
  <Section position="7" start_page="181" end_page="182" type="evalu">
    <SectionTitle>
5.2.2 Results
</SectionTitle>
    <Paragraph position="0"> Table 2 shows the results of running all three algorithms, evaluating against five criteria. Notice that for each algorithm, for the criterion that it optimizes it is the best algorithm. That is, the Labelled Tree Algorithm is the best for the Labelled Tree Rate, the Labelled Recall Algorithm is the best for the Labelled Recall Rate, and the Bracketed Recall Algorithm is the best for the Bracketed Recall Rate.</Paragraph>
    <Paragraph position="1"> Matching parsing algorithms to evaluation criteria is a powerful technique that can be used to improve performance. In particular, the Labelled Recall Algorithm can improve performance versus the Labelled Tree Algorithm on the Consistent Brackets, Labelled Recall, and Bracketed Recall criteria.</Paragraph>
    <Paragraph position="2"> Similarly, the Bracketed Recall Algorithm improves performance (versus Labelled Tree) on Consistent Brackets and Bracketed Recall criteria. Thus, these algorithms improve performance not only on the measures that they were designed for, but also on related criteria.</Paragraph>
    <Paragraph position="3"> Furthermore, in some cases these techniques can make parsing fast when it was previously impractical. We have used the technique outlined in this paper in other work (Goodman, 1996) to efficiently parse the DOP model; in that model, the only previously known algorithm which summed over all the  possible derivations was a slow Monte Carlo algorithm (Bod, 1993). However, by maximizing the Labelled Recall criterion, rather than the Labelled Tree criterion, it was possible to use a much simpler algorithm, a variation on the Labelled Recall Algorithm. Using this technique, along with other optimizations, we achieved a 500 times speedup.</Paragraph>
    <Paragraph position="4"> In future work we will show the surprising result that the last element of Table 3, maximizing the Bracketed Tree criterion, equivalent to maximizing performance on Consistent Brackets Tree (Zero Crossing Brackets) Rate in the binary branching case, is NP-complete. Furthermore, we will show that the two algorithms presented, the Labelled Recall Algorithm and the Bracketed Recall Algorithm, are both special cases of a more general algorithm, the General Recall Algorithm. Finally, we hope to extend this work to the n-ary branching case.</Paragraph>
  </Section>
class="xml-element"></Paper>