<?xml version="1.0" standalone="yes"?>
<Paper uid="P98-1072">
  <Title>Semantic-Head Based Resolution of Scopal Ambiguities*</Title>
  <Section position="7" start_page="436" end_page="436" type="evalu">
    <SectionTitle>
5 Implementation and Evaluation
</SectionTitle>
    <Paragraph position="0"> The resolution algorithm described in Section 4 has been implemented in Verbmobil, a system which translates spoken German and Japanese into English (Bub et al., 1997). The under-specified semantic representation technique we have used in this paper reflects the core semantic part of the Verbmobil Interface Term, VIT (Bos et al., 1998). The aim of VIT is to describe a consistent interface structure between the different language analysis modules within Verbmobil. Thus, in contrast to our USR, VIT is a representation that encodes all the linguistic information of an utterance; in addition to the USR semantic structure of Sectiom 2, the Verbmobil Interface Term contains prosodic, syntactic, and discourse related information.</Paragraph>
    <Paragraph position="1"> In order to evaluate the algorithm, the results of the pluggings obtained for four dialogues in the Verbmobil test set were checked (Table 1).</Paragraph>
    <Paragraph position="2"> We only consider utterances for which the VITs contain more than two holes: The number of scope-bearing operators is the number of holes minus one. Thus, a VIT with one hole only trivially contains the top hole of the utterance (i.e., the hole for the sentence mood predicate; introduced by the main verb).</Paragraph>
    <Paragraph position="3"> A VIT with two holes contains the top hole and the hole for one scope-taking element. However, the mood-predicate will always have scope over the remaining proposition, so resolution is still trivial.</Paragraph>
    <Paragraph position="4">  The dialogues evaluated are identified as three of the &amp;quot;Blaubeuren&amp;quot; dialogues (B1, B2, and BT) and one of the &amp;quot;Reithinger-Herweg-Quantz&amp;quot; dialogues (RHQ1). These four together form the standard test-set for the German language modules of the Verbmobil system. For VITs with three or more holes, we have true ambiguities. Column 3 gives the number of utterances with no ambiguity (&lt; 2 holes), the columns following look at the ambiguous sentences. Most commonly the utterances contained one true ambiguity (3 holes, as in Fig. 2). Utterances with more than two ambiguities (&gt; 5 holes) are rare and have been grouped together.</Paragraph>
    <Paragraph position="5"> Even though the algorithm is fairly straightforward, resolution based on semantic argument structure fares quite well. Only 64 (28%) of the 228 utterances are truely ambiguous (i.e., contain more than two holes). The default scoping introduced by the algorithm is the preferred one for 80% of the ambiguous utterances, leaving errors in just 13 (5.7%) of the utterances overall.</Paragraph>
    <Paragraph position="6"> Looking closer at these cases, the reasons for the failures divide as: the relative scope of two particles did not conform to the c-command structure assigned by syntax (one case); an indefinite noun phrase should have received wide scope (3), or narrow scope (1); an adverb should have had wide scope (3); combination of (a modal) verb movement and negated question (1); technical construction problem in VIT (4).</Paragraph>
    <Paragraph position="7"> The resolution algorithm has been implemented in Verbmobil in both the German semantic processing (Bos et al., 1996) and the (substantially smaller) Japanese one (Gamb~ick et al., 1996). Evaluating the performance of the resolution algorithm on the standard test suite for the Japanese parts of Verbmobil (the &amp;quot;RDSI&amp;quot; reference dialogue), we found that only 7 of the 36 sentences in the dialogue contained more than two holes. All but one of the ambiguities were correctly resolved by the algorithm. Even though the number of sentences tested certainly is too small to draw any real conclusions from, the correctness rate still indicates that the algorithm is applicable also to Japanese.</Paragraph>
  </Section>
class="xml-element"></Paper>