<?xml version="1.0" standalone="yes"?>
<Paper uid="P06-2042">
  <Title>Detection of Quotations and Inserted Clauses and its Application to Dependency Structure Analysis in Spontaneous Japanese Ryoji Hamabe DD</Title>
  <Section position="6" start_page="326" end_page="328" type="evalu">
    <SectionTitle>
4 Experiments and Discussion
</SectionTitle>
    <Paragraph position="0"> For experimental evaluation, we used the transcriptions of 188 talks in the CSJ, which contain 6,255 quotations and 818 inserted clauses. We used 20 talks for testing. The test data included 643 quotations and 76 inserted clauses. For training, we used 168 talks excluding the test data to conduct the open test and all the 188 talks to conduct the closed test.</Paragraph>
    <Paragraph position="1"> First, we detected sentence boundaries by using the method (Shitaoka et al., 2004) and analyzed the dependency structure of each sentence by the method described in Section 3.1 without using information on quotations and inserted clauses. We obtained an F-measure of 85.9 for the sentence boundary detection, and the baseline accuracy of the dependency structure analysis was 77.7% for the open test and 86.5% for the closed test.</Paragraph>
    <Paragraph position="2">  (a) Results of clause boundary detection The results obtained by the method described in Section 3.2 are shown in Table 2. The table shows five kinds of results: AF results obtained without dependency structure (in the first chunking step) AF results obtained with dependency structure analyzed for the open test (in the second chunking step) AF results obtained with dependency structure analyzed for the closed test (in the second chunking step) AF results obtained with manually annotated dependency structure (in the second chunking step) AF the rate that the ends of clauses are detected correctly These results indicate that around 90% of quotations were detected correctly, and the boundary detection accuracy of quotations was improved by using automatically analyzed dependency structure. We found that features (4) and (5) in Section 3.2 obtained from automatically analyzed dependency structure contributed to the improvement. In the following example, a part of the quotation &amp;quot; wMMtasMT&amp;quot; (my good virtue) was erroneously detected as a quotation in the first chunking step. But, in the second chunking step, automatically analyzed dependency structure contributed to detection of the correct part &amp;quot;\x wMMtasMT&amp;quot; (this is my good virtue) as a quotation.</Paragraph>
    <Paragraph position="4"> We also found that the boundary detection accuracy of quotations was significantly improved by using manually annotated dependency structure.</Paragraph>
    <Paragraph position="5"> This indicates that the boundary detection accuracy of quotations improves as the accuracy of dependency structure analysis improves.</Paragraph>
    <Paragraph position="6"> By contrast, only a few inserted clauses were detected even if dependency structures were used.</Paragraph>
    <Paragraph position="7"> Most of the ends of the inserted clauses were detected incorrectly as sentence boundaries. The main reason for this is our method could not distinguish between the ends of the inserted clauses and those of the sentences, since the same words often appeared at the ends of both, and it was difficult  tained with clause boundaries (sentence boundaries automatically detected) Without boundaries of quotations open 77.7% and inserted clauses closed 86.5% With boundaries of quotations and open 78.5% inserted clauses (automatically detected) closed 86.6% With boundaries of quotations and open 79.4% inserted clauses (correct) closed 87.4% to learn the difference between them even though our method used features based on acoustic information. null (b) Dependency structure analysis results We investigated the accuracies of dependency structure analysis obtained when the automatically or manually detected boundaries of quotations and inserted clauses were used. The results are shown in Table 3. Although the accuracy of detecting the boundaries of quotations and inserted clauses using automatically analyzed dependency structure was not high, the accuracy of dependency structure analysis was improved by 0.7% absolute for the open test. This shows that the model for dependency structure analysis could robustly learn useful information on clause boundaries even if errors were included in the results of clause boundary detection. In the following example, for instance, &amp;quot;%ptZo`O&amp;quot; (to go out with its face stuck) was correctly detected as a quotation in the first chunking step. Then, the initial inappropriate modifiee &amp;quot;QoVo, oboe-te-ki-te&amp;quot; (learn) of the bunsetsu inside the quotation &amp;quot; p, hasan-de&amp;quot; (stick) was correctly modified to the bunsetsu inside the quotation &amp;quot;Zo`OqMO, de-te-shimau-to-iu&amp;quot; (to go) by using the automatically detected boundary of the quotation.</Paragraph>
    <Paragraph position="8">  boundaries are given We investigated the clause boundary detection accuracy of quotations and inserted clauses and the dependency accuracy when correct sentence boundaries were given manually. The results are shown in Tables 4 and 5, respectively.</Paragraph>
    <Paragraph position="9"> When correct sentence boundaries were given, the accuracy of clause detection and dependency structure analysis was improved significantly. Table 4 shows that the boundary detection accuracy of inserted clauses as well as that of quotations was significantly improved by using information of dependencies. Table 5 indicates that when using automatically detected clause boundaries, the accuracy of dependency structure analysis was improved by 0.7% for the open test, and it was further improved by using correct clause boundaries.</Paragraph>
    <Paragraph position="10"> These experimental results show that detecting the boundaries of quotations and inserted clauses as well as sentence boundaries is sensitive to the accuracy of dependency structure analysis and the improvements of the boundary detection of quotations and inserted clauses contribute to improvement of dependency structure analysis. Especially, the difference between Table3 and 5 shows that the sentence boundary detection accuracy is more sensitive to the accuracy of dependency structure analysis than the boundary detection accuracy of quotations and inserted clauses. This indicates that sentence boundaries rather than quotations and inserted clauses should be manually examined first to effectively improve the accuracy of dependency structure analysis in a semi-automatic way.</Paragraph>
  </Section>
class="xml-element"></Paper>