<?xml version="1.0" standalone="yes"?>
<Paper uid="N01-1016">
  <Title>Edit Detection and Parsing for Transcribed Speech</Title>
  <Section position="2" start_page="0" end_page="1" type="intro">
    <SectionTitle>
1 Introduction
</SectionTitle>
    <Paragraph position="0"> While signi cant e ort has been expended on the parsing of written text, parsing speech has received relatively little attention. The comparative neglect of speech (or transcribed speech) is understandable, since parsing transcribed speech presents several problems absent in regular text: \um&amp;quot;s and \ah&amp;quot;s (or more formally, lled pauses), frequent use of parentheticals (e.g., \you know&amp;quot;), ungrammatical constructions, and speech repairs (e.g., \Why didn't he, why didn't she stay home?&amp;quot;).</Paragraph>
    <Paragraph position="1"> In this paper we present and evaluate a simple two-pass architecture for handling the problems of parsing transcribed speech. The rst pass tries to identify which of the words in the string are edited (\why didn't he,&amp;quot; in the above example). These words are removed from the string given to the second pass, an already existing statistical parser trained on a transcribed speech This research was supported in part by NSF grant LIS SBR 9720368 and by NSF ITR grant 20100203.</Paragraph>
    <Paragraph position="2"> corpus. (In particular, all of the research in this paper was performed on the parsed \Switchboard&amp;quot; corpus as provided by the Linguistic</Paragraph>
    <Section position="1" start_page="0" end_page="1" type="sub_section">
      <SectionTitle>
Data Consortium.)
</SectionTitle>
      <Paragraph position="0"> This architecture is based upon a fundamental assumption: that the semantic and pragmatic content of an utterance is based solely on the unedited words in the word sequence.</Paragraph>
      <Paragraph position="1"> This assumption is not completely true. For example, Core and Schubert [8] point to counterexamples such as \have the engine take the oranges to Elmira, um, I mean, take them to Corning&amp;quot; where the antecedent of \them&amp;quot; is found in the EDITED words. However, we believe that the assumption is so close to true that the number of errors introduced by this assumption is small compared to the total number of errors made by the system.</Paragraph>
      <Paragraph position="2"> In order to evaluate the parser's output we compare it with the gold-standard parse trees.</Paragraph>
      <Paragraph position="3"> For this purpose a very simple third pass is added to the architecture: the hypothesized edited words are inserted into the parser output (see Section 3 for details). To the degree that our fundamental assumption holds, a \real&amp;quot; application would ignore this last step.</Paragraph>
      <Paragraph position="4"> This architecture has several things to recommend it. First, it allows us to treat the editing problem as a pre-process, keeping the parser unchanged. Second, the major clues in detecting edited words in transcribed speech seem to be relatively shallow phenomena, such as repeated word and part-of-speech sequences. The kind of information that a parser would add, e.g., thenodedominatingtheEDITED node, seems much less critical.</Paragraph>
      <Paragraph position="5"> Note that of the major problems associated with transcribed speech, we choose to deal with only one of them, speech repairs, in a special fashion. Our reasoning here is based upon what one might and might not expect from a second-pass statistical parser. For example, ungrammaticality in some sense is relative, so if the training corpus contains the same kind of ungrammatical examples as the testing corpus, one would not expect ungrammaticality itself to be a show stopper. Furthermore, the best statistical parsers [3,5] do not use grammatical rules, but rather de ne probability distributions over all possible rules.</Paragraph>
      <Paragraph position="6"> Similarly, parentheticals and lled pauses exist in the newspaper text these parsers currently handle, albeit at a much lower rate. Thus there is no particular reason to expect these constructions to have a major impact.</Paragraph>
      <Paragraph position="7">  This leaves speech repairs as the one major phenomenon not present in written text that might pose a major problem for our parser. It is for that reason that we have chosen to handle it separately. The organization of this paper follows the architecture just described. Section 2 describes the rst pass. We present therein a boosting model for learning to detect edited nodes (Sections 2.1 { 2.2) and an evaluation of the model as a stand-alone edit detector (Section 2.3).</Paragraph>
      <Paragraph position="8"> Section 3 describes the parser. Since the parser is that already reported in [3], this section simply describes the parsing metrics used (Section 3.1), the details of the experimental setup (Section 3.2), and the results (Section 3.3).</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>