<?xml version="1.0" standalone="yes"?>
<Paper uid="N04-4018">
  <Title>Improving Automatic Sentence Boundary Detection with Confusion Networks</Title>
  <Section position="3" start_page="0" end_page="0" type="relat">
    <SectionTitle>
2 Related Work
</SectionTitle>
    <Paragraph position="0"/>
    <Section position="1" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
2.1 Sentence Boundary Detection
</SectionTitle>
      <Paragraph position="0"> Previous work on sentence boundary detection for automatically recognized words has focused on the prosodic features and words of the single best recognizer output (Shriberg et al., 2000). That system had an HMM structure that integrates hidden event language modeling with prosodic decision tree outputs (Breiman et al., 1984). The HMM states predicted at each word boundary consisted of either a sentence or non-sentence boundary classification, each of which received a confidence score. Improvements to the hidden event framework have included interpolation of multiple language models (Liu et al., 2003).</Paragraph>
      <Paragraph position="1"> A related model has been used to investigate punctuation prediction for multiple hypotheses in a speech recognition system (Kim and Woodland, 2001). That system found improvement in punctuation prediction when rescoring using the classification tree prosodic feature model, but it also introduced a small increase in word error rate. More recent work has also implemented a similar model, but used prosodic features in a neural net instead of a decision tree (Srivastava and Kubala, 2003).</Paragraph>
      <Paragraph position="2"> A maximum entropy model that included pause information was used in (Huang and Zweig, 2002). Both finite-state models and neural nets have been investigated for prosodic and lexical feature combination in (Christensen et al., 2001).</Paragraph>
    </Section>
    <Section position="2" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
2.2 Confusion Networks
</SectionTitle>
      <Paragraph position="0"> Confusion networks are a compacted representation of word lattices that have strictly ordered word hypothesis slots (Mangu et al., 2000). The complexity of lattice representations is reduced to a simpler form that maintains all possible paths from the lattice (and more), but transforms the space to a series of slots which each have word hypotheses (and null arcs) derived from the lattice and associated posterior probabilities. Confusion networks may also be constructed from an N-best list, which is the case for these experiments. Confusion networks are used to optimize word error rate (WER) by selecting the word with the highest probability in each particular slot.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>