<?xml version="1.0" standalone="yes"?>
<Paper uid="P98-2135">
  <Title>Discourse Cues for Broadcast News Segmentation</Title>
  <Section position="3" start_page="0" end_page="0" type="metho">
    <SectionTitle>
2. Broadcast News Analysis
</SectionTitle>
    <Paragraph position="0"> Human communication is characterized by distinct discourse structure (Grosz and Sidner 1986) which is used for a variety of purposes including managing interaction between participants, mitigating limited attention, and signaling topic shifts. In processing genre such as technical or journalistic texts, programs can take advantage of explicit discourse cues (e.g., &amp;quot;the first&amp;quot;, &amp;quot;the most important&amp;quot;) to perform tasks such as summarization (Paice 1981). Our initial inability to segment topics in closed caption news text using thesaurus based subject assessments (Liddy and Myaeng 1992) motivated an investigation of explicit turn taking signals (e.g., anchor to reporter handoff). We analyzed programs (e.g., CNN PrimeNews) from an over one year corpus of closed caption texts with the intention of creating models of discourse and other cues for segmentation.</Paragraph>
  </Section>
  <Section position="4" start_page="0" end_page="819" type="metho">
    <SectionTitle>
(CNN Prime News, August 17, 1997)
</SectionTitle>
    <Paragraph position="0"> While human captioners employ standard cues to signal discourse shifts in the closed caption stream (e.g., &amp;quot;&gt;&gt;&amp;quot; is used to signal a speaker shift whereas &amp;quot;&gt;&gt;&gt;&amp;quot; signals a subject change), these can be erroneous, incomplete, or inconsistent. Figure 1 illustrates a typical excerpt from our corpus. Our creation of a gold standard corpus of a variety of broadcast sources indicates that transcription word error rates range from 2% for pre-recorded programs such as 60 Minutes news magazine to 20% for live transcriptions (including errors of insertion, deletion, and transposition). This noisy data complicates robust story segmentation.</Paragraph>
    <Section position="1" start_page="819" end_page="819" type="sub_section">
      <SectionTitle>
2.1 News Story Discourse Structure
</SectionTitle>
      <Paragraph position="0"> Broadcast news has a prevalent structure with often explicit cues to signal story shifts. For example, analysis of the structure of ABC World News Tonight indicates: * broadcasts start and end with the anchor * reporter segments are preceded by an introductory anchor segment and together they form a single story * commercials serve as story boundaries Similar but unique structure is also prevalent in many other news programs such as CNN Prime News (See Figure 1) or MS-NBC. For example, the structure for the Jim Lehrer News Hour provides not only segmentation information but also content information for each segment. Thus, the order of stories is consistently:  Recovering this structure would enable a user to view the four minute opening summary, retrieve daily news summaries, preview and retrieve major stories, or browse a video table of contents, with or without commercials.</Paragraph>
    </Section>
    <Section position="2" start_page="819" end_page="819" type="sub_section">
      <SectionTitle>
2.2 Discourse Cues and Named Entities
</SectionTitle>
      <Paragraph position="0"> Manual and semi-automated analysis of our news corpora reveals that regular cues are used to signal these shifts in discourse, although this structure varies dramatically from source to source. For example, CNN discourse cues can be classified into the following categories (examples from 8/18/97):  The regularity of these discourse cues from broadcast to broadcast provides an effective foundation for discourse-based segmentation routines. We have similarly discovered regular discourse cues in other news programs. For example, anchor/reporter and reporter/anchor handoffs in CNN Prime News or ABC News and other network programs are identified through pattern matching of strings such as:</Paragraph>
      <Paragraph position="2"> The pairs of words in parentheses correspond to the reporter's first and last names. Combining the handoffs with structural cues, such as knowing that the first and last speaker in the program will be the anchor, allow us differentiate anchor segments from reporter segments. By preprocessing the closed caption text with a part of speech tagger and named entity detector (Aberdeen et al. 1995) retrained on closed captions, we generalize search of text strings to the following class of patterns:</Paragraph>
    </Section>
  </Section>
  <Section position="5" start_page="819" end_page="820" type="metho">
    <SectionTitle>
3. Computational Implementation
</SectionTitle>
    <Paragraph position="0"> Our discourse cue story segmentor has been implemented in the context of a multimedia (closed captioned text, audio, video) analysis system for web based broadcast news navigation. We employ a finite state machine to represent discourse states such as an anchor, reporter, or advertisting segment (See Figure 2). We further enhance these with multimedia cues (e.g. detected Silence, black or logo keyframes) and temporal knowledge (indicated as time in Figure 2). For example, from statistical analysis of CNN Prime News Programs, we know that weather segments appear on average 18 minutes after the start of the news.</Paragraph>
    <Paragraph position="1">  After segmentation, the user is presented with a hierarchical navigation space of the news which enables search and retrieval of segmented stories or browsing stories by date, topic, named entity or keyword (see Figure 3). This is MITRE's  We leverage the story segments and extracted named entities to select the sentence with the most named entities to serve as a single sentence summary of a given segment. Story structure is also useful for multimedia summarization. For example, we can select key frames or key words from the substructure which will likely contain the most meaningful content (e.g., an reporter segment within an anchor segment).</Paragraph>
  </Section>
class="xml-element"></Paper>