<?xml version="1.0" standalone="yes"?>
<Paper uid="P97-1049">
  <Title>Hierarchical Non-Emitting Markov Models</Title>
  <Section position="4" start_page="0" end_page="381" type="intro">
    <SectionTitle>
2 Background
</SectionTitle>
    <Paragraph position="0"> Here we review the basic Markov model and the interpolated Markov model, and establish their equivalence. null A basic Markov model C/ = (A,n,6,) consists of an alphabet A, a model order n, n &gt; 0, and the state transition probabilities 6, : A n x A ---* \[0, 1\]. With probability 6,(y\[zn), a Markov model in the state z '~ will emit the symbol y and transition to the state z'~y. Therefore, the probability Prn(ZtlX t-1 , C/) assigned by an order n basic Markov model C/ to a symbol z' in the history z t-1 depends only on the last n symbols of the history.</Paragraph>
    <Paragraph position="2"> An interpolated Markov model C/ = (A,n,A,6) consists of a finite alphabet A, a maximal model order n, the state transition probabilities 6 = 60 ... 6,,</Paragraph>
    <Paragraph position="4"> The probability assigned by an interpolated model is a linear combination of the probabilities assigned by all the lower order Markov models.</Paragraph>
    <Paragraph position="6"> where )q(z i) = 0 for i &gt; n, and and therefore p~(z, lzt-1, C/) ,-7 = pC/(ztlzt_,~,C/), ie., the prediction depends only on the last n symbols of the history.</Paragraph>
    <Paragraph position="7"> In the interpolated model, the interpolation parameters smooth the conditional probabilities estimated from longer histories with those estimated from shorter histories (:lelinek and Mercer, 1980).</Paragraph>
    <Paragraph position="8"> Longer histories support stronger predictions, while shorter histories have more accurate statistics. Interpolating the predictions from histories of different lengths results in more accurate predictions than can be obtained from any fixed history length.</Paragraph>
    <Paragraph position="9"> A quick glance at the form of (2) and (1) reveals the fundamental simplicity of the interpolated Markov model. Every interpolated model C/ is equivalent to some basic Markov model C/' (temma 2.1), and every basic Markov model C/ is equivalent to some interpolated context model C/' (lemma 2.2).</Paragraph>
    <Paragraph position="11"> Proof. We may convert the interpolated model C/ into a basic model C/' of the same model order n, simply by setting 6&amp;quot;(ylz n) equal to pc(y\[z n, C/) for all states z n E A n and symbols y 6 A. \[\] Lemma 2.2 VC/ ~C/t vzT 6 A* \[pc(zTIC/',T) = pm(xT\]C/,T)\] Proof. Every basic model is equivalent to an interpolated model whose interpolation values are unity for states of order n. \[\] The lemmas suffice to establish the following theorem. null Theorem 1 The class of interpolated Markov models is equivalent to the class of basic Markov models. Proof. By lemmas 2.1 and 2.2. f&amp;quot;l A similar argument applies to the backoff model. Every backoff model can be converted into an equivalent basic model, and every basic model is a backoff model.</Paragraph>
  </Section>
class="xml-element"></Paper>