<?xml version="1.0" standalone="yes"?>
<Paper uid="W05-0905">
  <Title>Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, pages 33-40, Ann Arbor, June 2005. c(c)2005 Association for Computational Linguistics Evaluating Automatic Summaries of Meeting Recordings</Title>
  <Section position="3" start_page="33" end_page="34" type="intro">
    <SectionTitle>
2 Description of the Summarization
</SectionTitle>
    <Paragraph position="0"/>
    <Section position="1" start_page="33" end_page="33" type="sub_section">
      <SectionTitle>
Approaches
2.1 Maximal Marginal Relevance (MMR)
</SectionTitle>
      <Paragraph position="0"> MMR (Carbonell and Goldstein, 1998) uses the vector-space model of text retrieval and is particularly applicable to query-based and multi-document summarization. The MMR algorithm chooses sentences via a weighted combination of queryrelevance and redundancy scores, both derived using cosine similarity. The MMR score ScMMR(i)for a given sentence Si in the document is given by</Paragraph>
      <Paragraph position="2"> where D is the average document vector, Summ is the average vector from the set of sentences already selected, and l trades off between relevance and redundancy. Sim is the cosine similarity between two documents.</Paragraph>
      <Paragraph position="3"> This implementation of MMR uses lambda annealing so that relevance is emphasized while the summary is still short and minimizing redundancy is prioritized more highly as the summary lengthens.</Paragraph>
    </Section>
    <Section position="2" start_page="33" end_page="33" type="sub_section">
      <SectionTitle>
2.2 Latent Semantic Analysis (LSA)
</SectionTitle>
      <Paragraph position="0"> LSA is a vector-space approach which involves projecting the original term-document matrix to a reduced dimension representation. It is based on the singular value decomposition (SVD) of an m x n term-document matrix A, whose elements Aij represent the weighted term frequency of term i in document j. In SVD, the term-document matrix is decomposed as follows: A = USV T where U is an mxn matrix of left-singular vectors, S is an n x n diagonal matrix of singular values, and V is the nxn matrix of right-singular vectors. The rows of V T may be regarded as defining topics, with the columns representing sentences from the document. Following Gong and Liu (Gong and Liu, 2001), summarization proceeds by choosing, for each row in V T, the sentence with the highest value. This process continues until the desired summary length is reached.</Paragraph>
      <Paragraph position="1"> Two drawbacks of this method are that dimensionality is tied to summary length and that good sentence candidates may not be chosen if they do not &amp;quot;win&amp;quot; in any dimension (Steinberger and JeVzek, 2004). The authors in (Steinberger and JeVzek, 2004) found one solution, by extracting a single LSA-based sentence score, with variable dimensionality reduction.</Paragraph>
      <Paragraph position="2"> We address the same concerns, following the Gong and Liu approach, but rather than extracting the best sentence for each topic, the n best sentences are extracted, with n determined by the corresponding singular values from matrix S. The number of sentences in the summary that will come from the first topic is determined by the percentage that the largest singular value represents out of the sum of all singular values, and so on for each topic. Thus, dimensionality reduction is no longer tied to summary length and more than one sentence per topic can be chosen. Using this method, the level of dimensionality reduction is essentially learned from the data.</Paragraph>
    </Section>
    <Section position="3" start_page="33" end_page="34" type="sub_section">
      <SectionTitle>
2.3 Feature-Based Approaches
</SectionTitle>
      <Paragraph position="0"> Feature-based classification approaches have been widely used in text and speech summarization, with positive results (Kupiec et al., 1995). In this work we combined textual and prosodic features, using Gaussian mixture models for the extracted and nonextracted classes. The prosodic features were the mean and standard deviation of F0, energy, and duration, all estimated and normalized at the wordlevel, then averaged over the utterance. The two lexical features were both TFIDF-based: the average and the maximum TFIDF score for the utterance.</Paragraph>
      <Paragraph position="1"> For our second feature-based approach, we derived single LSA-based sentence scores (Steinberger and JeVzek, 2004) to complement the six features described above, to determine whether such an LSA sentence score is beneficial in determining sentence importance. We reduced the original term-document matrix to 300 dimensions; however, Steinberger and JeVzek found the greatest success in their work by reducing to a single dimension (Steinberger, personal communication). The LSA sentence score was obtained using:</Paragraph>
      <Paragraph position="3"> where v(i,k) is the kth element of the ith sentence vector and s(k) is the corresponding singular value.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>