<?xml version="1.0" standalone="yes"?>
<Paper uid="W06-1644">
  <Title>Style &amp; Topic Language Model Adaptation Using HMM-LDA</Title>
  <Section position="8" start_page="379" end_page="380" type="concl">
    <SectionTitle>
6 Summary and Conclusions
</SectionTitle>
    <Paragraph position="0"> In this paper, we have shown how to leverage context-dependent state and topic labels, such as the ones generated by the HMM-LDA model, to construct better language models for lecture transcription and extend topic models beyond traditional unigrams. Although the WER of the top recognizer hypotheses exceeds 45%, by dynamically updating the mixture weights to model the topic substructure within individual lectures, we are able to reduce the test set perplexity and WER by over 16% and 2.4%, respectively, relative to the combined Lectures and Textbook (L+T) baseline.</Paragraph>
    <Paragraph position="1"> Although we primarily focused on lecture transcription in this work, the techniques extend to language modeling scenarios where exactly matched training data are often limited or nonexistent. Instead, we have to rely on appropriate combination of models derived from partially matched data. HMM-LDA and related techniques show great promise for finding structure in unlabeled data, from which we can build more sophisticated models.</Paragraph>
    <Paragraph position="2"> The experiments in this paper combine models primarily through simple linear interpolation. As motivated in section 5.2, allowing for context-dependent interpolation weights based on topic  labels may yield significant improvement for both perplexity and WER. Thus, in future work, we would like to study algorithms for automatically learning appropriate context-dependent interpolation weights. Furthermore, we hope to improve the convergence properties of the dynamic adaptation scheme at the start of lectures and across topic transitions. Lastly, we would like to extend the LDA framework to support speaker-specific adaptation and apply the resulting topic distributions to lecture segmentation.</Paragraph>
  </Section>
class="xml-element"></Paper>