<?xml version="1.0" standalone="yes"?>
<Paper uid="P98-1083">
  <Title>Using Decision Trees to Construct a Practical Parser</Title>
  <Section position="2" start_page="0" end_page="505" type="intro">
    <SectionTitle>
1 Introduction
</SectionTitle>
    <Paragraph position="0"> Conventional parsers with practical levels of performance require a number of sophisticated rules that have to be hand-crafted by human linguists. It is time-consunaing and cumbersome to naaintain these rules for two reasons.</Paragraph>
    <Paragraph position="1"> * The rules are specific to the application domain.</Paragraph>
    <Paragraph position="2"> * Specific rules handling collocational expressions create side effects. Such rules often deteriorate t, he overall performance of the parser.</Paragraph>
    <Paragraph position="3"> The stochastic approach, on the other hand, has the potential to overcome these difficulties. Because it. induces stochastic rules to maximize overall performance against training data, it not only adapts to any application domain but. also may avoid over-fitting to the data. In the late 80s and early 90s, the induction and parameter estimation of probabilistic context free grammars (PCFGs) from corpora were intensively studied. Because these grammars comprise only nonterminal and part-of-speech tag symbols, their performances were not enough to be used in practical applications (Charniak, 1993). A broader range of information, in particular lexical information, was found to be essential in disambiguating the syntactic structures of real-world sentences. SPATTER (Magerman, 1995) augmented the pure PCFG by introducing a number of lexical attributes.</Paragraph>
    <Paragraph position="4"> The parser controlled applications of each rule by using the lexical constraints induced by decision tree algorithm (Quinlan, 1993). The SPATTER parser attained 87% accuracy and first made stochastic parsers a practical choice. The other type of high-precision parser, which is based on dependency analysis was introduced by Collins (Collins, 1996). Dependency analysis first segments a sentence into syntactically meaningful sequences of words and then considers the modification of each segment. Collins' parser computes the likelihood that each segment modifies the other (2 term relation) by using large corpora. These modification probabilities are conditioned by head words of two segments, distance between the two segments and other syntactic features. Although these two parsers have shown similar performance, the keys of their success are slightly different. SPATTER parser performance greatly depends on the feature selection ability of the decision tree algorithm rather than its linguistic representation. On the other hand, dependency analysis plays an essential role in Collins' parser for efficiently extracting information from corpora.</Paragraph>
    <Paragraph position="5"> In this paper, we describe practical Japanese dependency parsers that uses decision trees. In the Japanese language, dependency analysis has been shown to be powerful because segment (bunsetsu) order in a sentence is relatively free compared to European languages..Japanese dependency parsers generally proceed in three steps.</Paragraph>
    <Paragraph position="6">  1. Segment a sentence into a sequence of bunsetsu.</Paragraph>
    <Paragraph position="7"> 2. Prepare a modification matrix, each value of which represents how one bunsetsu is likely to modify another.</Paragraph>
    <Paragraph position="8"> 3. Find optimal modifications in a sentence by a  dynamic programming technique.</Paragraph>
    <Paragraph position="9"> The most difficult part is the second; how to construct a sophisticated modification matrix. With conventional Japanese parsers, the linguist nmst classify the bunsetsu and select appropriate features to compute modification values. The parsers thus suffer from application domain diversity and the side effects of specific rules.</Paragraph>
    <Paragraph position="10">  Stochastic dependency parsers like Collins', on the other hand, define a set of attributes for conditioning the modification probabilities. The parsers consider all of the attributes regardless of bunsetsu type. These methods can encompass only a small number of features if the probabilities are to be precisely evaluated from finite number of data. Our decision tree method constructs a more sophisticated modification matrix. It automatically selects a sufficient number of significant attributes according to bunsetsu type. We can use arbitrary numbers of the attributes which potentially increase parsing accuracy. null Natural languages are full of exceptional and collocational expressions. It is difficult for machine learning algorithms, as well as human linguists, to judge whether a specific rule is relevant in terms of over-all performance. To tackle this problem, we test the mixture of sequentially generated decision trees. Specifically, we use the Ada-Boost algorithm (Freund and Schapire, 1996) which iteratively performs two procedures: 1. construct a decision tree based on the current data distribution and 2. updating the distribution by focusing on data that are not well predicted by the constructed tree. The final modification probabilities are computed by mixing all the decision trees according to their performance. The sequential decision trees gradually change from broad coverage to specific exceptional trees that. cannot be captured by a single general tree. In other words, the method incorporates not only general expressions but also infrequent specific ones.</Paragraph>
    <Paragraph position="11"> The rest of the paper is constructed as follows.</Paragraph>
    <Paragraph position="12"> Section 2 summarizes dependency analysis for the Japanese language. Section 3 explains our decision tree models that compute modification probabilities. Section 4 then presents experimental results obtained by using EDR Japanese annotated corpora.</Paragraph>
    <Paragraph position="13"> Finally, section 5 concludes the paper.</Paragraph>
  </Section>
class="xml-element"></Paper>