<?xml version="1.0" standalone="yes"?>
<Paper uid="C02-2003">
  <Title>Searching the Web by Voice</Title>
  <Section position="3" start_page="0" end_page="0" type="intro">
    <SectionTitle>
2 Trade-Offs in Language Modeling
</SectionTitle>
    <Paragraph position="0"> A speech recognition system uses a language model to determine the probability of different recognition hypotheses. For our application, there is a trade-off among three considerations: What fraction of the query traffic is covered by the vocabulary of the language model? How much predictive power does the language model provide? And what is the observed computational complexity of applying the language model during hypothesis search? At one extreme, a language model that simply used a list of the most frequent queries in their entirety would have the lowest coverage, but would provide the best predictive power within the covered queries (have the lowest per-query perplexity), and would be the least computationally expensive. At the other extreme, (Lau, 1998; Ng, 2000) report on experiments with sub-word n-gram language models, which have very high coverage, but rather low predictive power (high per-query perplexity).</Paragraph>
    <Paragraph position="1"> We experimented with various configurations of back-off word n-gram models (Katz, 1987; Jelinek, 1997). In our experience with commercially available speech recognition systems, we found that for a vocabulary size of 100,000 items, unigram models were the only computationally feasible choice, yielding close to real-time performance. When using the bigram model, the recognizer needed to spend several minutes processing each utterance to achieve accuracy as high as it achieved with the uni-gram model. Recognition with a bigram model was unacceptably slow even when we pruned the model by removing bigrams that provided little improvement in perplexity (Stolcke, 1998). For this reason, we explored a method to increase the predictive power of the unigram model by adding collocations to its vocabulary.</Paragraph>
  </Section>
class="xml-element"></Paper>