<?xml version="1.0" standalone="yes"?>
<Paper uid="P99-1069">
  <Title>Estimators for Stochastic &amp;quot;Unification-Based&amp;quot; Grammars*</Title>
  <Section position="2" start_page="0" end_page="0" type="intro">
    <SectionTitle>
1 Introduction
</SectionTitle>
    <Paragraph position="0"> Probabilistic methods have revolutionized computational linguistics. They can provide a systematic treatment of preferences in parsing. Given a suitable estimation procedure, stochastic models can be &amp;quot;tuned&amp;quot; to reflect the properties of a corpus. On the other hand, &amp;quot;Unification-Based&amp;quot; Grammars (UBGs) can express a variety of linguistically-important syntactic and semantic constraints. However, developing Stochastic &amp;quot;Unification-based&amp;quot; Grammars (SUBGs) has not proved as straight-forward as might be hoped.</Paragraph>
    <Paragraph position="1"> The simple &amp;quot;relative frequency&amp;quot; estimator for PCFGs yields the maximum likelihood parameter estimate, which is to say that it minimizes the Kulback-Liebler divergence between the training and estimated distributions.</Paragraph>
    <Paragraph position="2"> On the other hand, as Abney (1997) points out, the context-sensitive dependencies that &amp;quot;unification-based&amp;quot; constraints introduce render the relative frequency estimator suboptimal: in general it does not maximize the likelihood and it is inconsistent.</Paragraph>
    <Paragraph position="3"> * This research was supported by the National Science Foundation (SBR,-9720368), the US Army Research Office (DAAH04-96-BAA5), and Office of Naval Research (N00014-97-1-0249).</Paragraph>
    <Paragraph position="4"> Abney (1997) proposes a Markov Random Field or log linear model for SUBGs, and the models described here are instances of Abney's general framework. However, the Monte-Carlo parameter estimation procedure that Abney proposes seems to be computationally impractical for reasonable-sized grammars. Sections 3 and 4 describe two new estimation procedures which are computationally tractable. Section 5 describes an experiment with a small LFG corpus provided to us by Xerox PAaC. The log linear framework and the estimation procedures are extremely general, and they apply directly to stochastic versions of HPSG and other theories of grammar.</Paragraph>
  </Section>
class="xml-element"></Paper>