<?xml version="1.0" standalone="yes"?>
<Paper uid="P02-1025">
  <Title>A Study on Richer Syntactic Dependencies for Structured Language Modeling</Title>
  <Section position="2" start_page="0" end_page="0" type="intro">
    <SectionTitle>
1 Introduction
</SectionTitle>
    <Paragraph position="0"> The structured language model uses hidden parse trees to assign conditional word-level language model probabilities. As explained in (Chelba and Jelinek, 2000), Section 4.4.1, if the final best parse is used to be the only parse, the reduction in PPL --relative to a 3-gram baseline-- using the SLM's headword parametrization for word prediction is about 40%. The key to achieving this reduction is a good guess of the final best parse for a given sentence as it is being traversed left-to-right, which is much harder than finding the final best parse for the entire sentence, as it is sought by a regular statistical parser. Nevertheless, it is expected that techniques developed in the statistical parsing community that aim at recovering the best parse for an entire sentence, i.e. as judged by a human annotator, should also be productive in enhancing the performance of a language model that uses syntactic structure.</Paragraph>
    <Paragraph position="1"> The statistical parsing community has used various ways of enriching the dependency structure underlying the parametrization of the probabilistic model used for scoring a given parse tree (Charniak, 2000) (Collins, 1999). Recently, such models (Charniak, 2001) (Roark, 2001) have been shown to out-perform the SLM in terms of both PPL and WER on the UPenn Treebank and WSJ corpora, respectively.</Paragraph>
    <Paragraph position="2"> In (Chelba and Xu, 2001), a simple way of enriching the probabilistic dependencies in the CONSTRUCTOR component of the SLM also showed better PPL and WER performance; the simple modification to the training procedure brought the WER performance of the SLM to the same level with the best as reported in (Roark, 2001).</Paragraph>
    <Paragraph position="3"> In this paper, we present three simple ways of enriching the syntactic dependency structure in the SLM, extending the work in (Chelba and Xu, 2001).</Paragraph>
    <Paragraph position="4"> The results show that an improved parser (as measured by LP/LR) is indeed helpful in reducing the PPL and WER. Another remarkable fact is that for the first time a language model exploiting elemen-Computational Linguistics (ACL), Philadelphia, July 2002, pp. 191-198. Proceedings of the 40th Annual Meeting of the Association for tary syntactic dependencies obviates the need for interpolation with a 3-gram model in N-best rescoring. null</Paragraph>
  </Section>
class="xml-element"></Paper>