<?xml version="1.0" standalone="yes"?>
<Paper uid="H92-1026">
  <Title>Towards History-based Grammars: Using Richer Models for Probabilistic Parsing*</Title>
  <Section position="4" start_page="0" end_page="134" type="intro">
    <SectionTitle>
2. Motivation for History-based
Grammars
</SectionTitle>
    <Paragraph position="0"> One goal of a parser is to produce a grammatical interpretation of a sentence which represents the syntactic and semantic intent of the sentence. To achieve this goal, the parser must have a mechanism for estimating the coherence of an interpretation, both in isolation and in context. Probabilistic language models provide such a mechanism.</Paragraph>
    <Paragraph position="1"> A probabilistic language model attempts to estimate the probability of a sequence of sentences and their respective interpretations (parse trees) occurring in the language, &amp;quot;P(S1 T1 $2 T2 ... S~ T~).</Paragraph>
    <Paragraph position="2"> The difficulty in applying probabilistic models to natu- null ral language is deciding what aspects of the sentence and the discourse are relevant to the model. Most previous probabilistic models of parsing assume the probabilities of sentences in a discourse are independent of other sentences. In fact, previous works have made much stronger independence assumptions. The P-CFG model considers the probability of each constituent rule independent of all other constituents in the sentence. The Pearl \[10\] model includes a slightly richer model of context, allowing the probability of a constituent rule to depend upon the immediate parent of the rule and a part-of-speech tri-gram from the input sentence. But none of these models come close to incorporating enough context to disambiguate many cases of ambiguity.</Paragraph>
    <Paragraph position="3"> A significant reason researchers have limited the contextual information used by their models is because of the difficulty in estimating very rich probabilistic models of context. In this work, we present a model, the history-based grammar model, which incorporates a very rich model of context, and we describe a technique for estimating the parameters for this model using decision trees. The history-based grammar model provides a mechanism for taking advantage of contextual information from anywhere in the discourse history. Using decision tree technology, any question which can be asked of the history (i.e. Is the subject of the previous sentence animate? Was the previous sentence a question? etc.) can be incorporated into the language model.</Paragraph>
  </Section>
class="xml-element"></Paper>