<?xml version="1.0" standalone="yes"?>
<Paper uid="W04-0302">
  <Title>Stochastically Evaluating the Validity of Partial Parse Trees in Incremental Parsing</Title>
  <Section position="2" start_page="0" end_page="0" type="intro">
    <SectionTitle>
1 Introduction
</SectionTitle>
    <Paragraph position="0"> Real-time spoken language processing systems, such as simultaneous machine interpretation systems, are required to quickly respond to users' utterances. To fulfill the requirement, the system needs to understand spoken language at least incrementally (Allen et al., 2001; Inagaki and Matsubara, 1995; Milward and Cooper, 1994), that is, to analyze each input sentence from left to right and acquire the content.</Paragraph>
    <Paragraph position="1"> Several incremental parsing methods have been proposed to date (Costa et al., 2001; Haddock, 1987; Matsubara et al., 1997; Milward, 1995; Roark, 2001). These methods construct candidate partial parse trees for initial fragments of the input sentence on a word-by-word basis. However, these methods contain local ambiguity problems that partial parse trees representing valid syntactic relations can not be determined without using information from the rest of the input sentence.</Paragraph>
    <Paragraph position="2"> On the other hand, Marcus proposed a method of deterministically constructing valid partial parse trees by looking ahead several words (Marcus, 1980), while Kato et al. proposed an incremental parsing which delays the decision of valid partial parse trees (Kato et al., 2000). However, it is hard to say that these methods realize broad-coverage incremental parsing. The method in the literature (Marcus, 1980) uses lookahead rules, which are constructed by hand, but it is not clear whether broad coverage lookahead rules can be obtained. The incremental parsing in the literature (Kato et al., 2000), which is based on context free grammar, is infeasible to deal with large scale grammar, because the parser exhaustively searches all candidate partial parse trees in top-down fashion.</Paragraph>
    <Paragraph position="3"> This paper proposes a probabilistic incremental parser which evaluates the validity of partial parse trees. Our method extracts a grammar from a treebank, and the incremental parsing uses a beam-search strategy so that it realizes broad-coverage parsing. To resolve local ambiguity, the parser incrementally evaluates the validity of partial parse trees on a word-by-word basis, and delays the decision of which partial parse trees should be returned, until the validity for the partial parse tree becomes greater than a threshold. Our technique is effective for improving the accuracy of incremental parsing.</Paragraph>
    <Paragraph position="4"> This paper is organized as follows: The next section proposes a probabilistic incremental parser.</Paragraph>
    <Paragraph position="5"> Section 3 discusses the validity of partial parse tree constructed in incremental parsing. Section 4 proposes a method of incrementally evaluating the validity of partial parse tree. In section 5, we report an experimental evaluation of our method.</Paragraph>
  </Section>
class="xml-element"></Paper>