<?xml version="1.0" standalone="yes"?>
<Paper uid="W06-1638">
  <Title>Better Informed Training of Latent Syntactic Features</Title>
  <Section position="3" start_page="0" end_page="317" type="intro">
    <SectionTitle>
1 Introduction
</SectionTitle>
    <Paragraph position="0"> Treebanks never contain enough information; thus PCFGs estimated straightforwardly from the Penn Treebank (Bies et al., 1995) work only moderately well (Charniak, 1996). To address this problem, researchers have used heuristics to add more information. Eisner (1996), Charniak (1997), Collins (1997), and many subsequent researchers1 annotated every node with lexical features passed up from its &amp;quot;head child,&amp;quot; in order to more precisely reflect the node's &amp;quot;inside&amp;quot; contents. Charniak (1997) and Johnson (1998) annotated each node with its parent and grandparent nonterminals, to more precisely reflect its &amp;quot;outside&amp;quot; context. Collins (1996) split the sentence label S into two versions, representing sentences with and without subjects. He 1Not to mention earlier non-PCFG lexicalized statistical parsers, notably Magerman (1995) for the Penn Treebank.</Paragraph>
    <Paragraph position="1"> also modified the treebank to contain different labels for standard and for base noun phrases. Klein and Manning (2003) identified nonterminals that could valuably be split into fine-grained ones using hand-written linguistic rules. Their unlexicalized parser combined several such heuristics with rule markovization and reached a performance similar to early lexicalized parsers.</Paragraph>
    <Paragraph position="2"> In all these cases, choosing which nonterminals to split, and how, was a matter of art. Ideally such splits would be learned automatically from the given treebank itself. This would be less costly and more portable to treebanks for new domains and languages. One might also hope that the automatically learned splits would be more effective.</Paragraph>
    <Paragraph position="3"> Matsuzaki et al. (2005) introduced a model for such learning: PCFG-LA.2 They used EM to induce fine-grained versions of a given treebank's nonterminals and rules. We present models that similarly learn to propagate fine-grained features through the tree, but only in certain linguistically motivated ways. Our models therefore allocate a supply of free parameters differently, allowing more fine-grained nonterminals but less fine-grained control over the probabilities of rewriting them. We also present simple methods for deciding selectively (during training) which nonterminals to split and how.</Paragraph>
    <Paragraph position="4"> Section 2 describes previous work in finding hidden information in treebanks. Section 3 describes automatically induced feature grammars.</Paragraph>
    <Paragraph position="5"> We start by describing the PCFG-LA model, then introduce new models that use specific agreement patterns to propagate features through the tree.</Paragraph>
    <Paragraph position="6"> Section 4 describes annealing-like procedures for training latent-annotation models. Section 5 describes the motivation and results of our experiments. We finish by discussing future work and conclusions in sections 6-7.</Paragraph>
    <Paragraph position="7"> 2Probabilistic context-free grammar with latent annotations. null  Citation Observed data Hidden data Collins (1997) Treebank tree with head child annotated on each nonterminal No hidden data. Degenerate EM case.</Paragraph>
    <Paragraph position="8"> Lari and Young (1990) Words Parse tree Pereira and Schabes (1992) Words and partial brackets Parse tree Klein and Manning (2001) Part-of-speech tags Parse tree Chiang and Bikel (2002) Treebank tree Head child on each nonterminal Matsuzaki et al. (2005) Treebank tree Integer feature on each nonterminal null INHERIT model (this paper) Treebank tree and head child heuristics Integer feature on each nonterminal null</Paragraph>
  </Section>
class="xml-element"></Paper>