<?xml version="1.0" standalone="yes"?>
<Paper uid="A00-2025">
  <Title>Minimizing Word Error Rate in Textual Summaries of Spoken Language</Title>
  <Section position="10" start_page="189" end_page="190" type="evalu">
    <SectionTitle>
8 Discussion
</SectionTitle>
    <Paragraph position="0"> The most significant result of our experiments is, in our opinion, the fact that the trade-off between word and summary accuracy indeed leads to an optimal parameter setting for the creation of textual summaries for spoken language (Figure 2). Using a formula which emphasizes turns containing many high confidence scores leads to an average WER reduction of over 10% and to an average improvement in summary accuracy of over 15%, compared to the baseline of a standard MMR-based summary.</Paragraph>
    <Paragraph position="1"> Comparing our results to those reported in (Valenza et al., 1999), we find that their relative  scores on a turn basis WER reduction for summaries over full texts was considerably larger than ours (60% vs. 24%). We conjecture that reasons for this may be due to the different nature and quality of the confidence scores, and (not unrelated), to the different absolute WER of the two corpora (25% vs. 35%): in transcripts with higher WER, the confidence scores are usually less reliable (eft Table 1).</Paragraph>
    <Paragraph position="2"> Looking at the four audio recordings individually, we see that the improvements vary strongly across different recordings. We conjecture that one reason for this fact may be due to the high variation in the correlation between WER and confidence scores on a turn basis (Table 5). This would explain why, e.g., BACK'S improvements are much stronger than those of the BUCHANAN recording or why there are no improvements for the 19CENT recording. However, GRAY does improve despite its very low absolute correlation.</Paragraph>
  </Section>
class="xml-element"></Paper>