<?xml version="1.0" standalone="yes"?>
<Paper uid="P99-1030">
  <Title>Analysis System of Speech Acts and Discourse Structures Using Maximum Entropy Model*</Title>
  <Section position="4" start_page="234" end_page="235" type="relat">
    <SectionTitle>
3 Experiments and results
</SectionTitle>
    <Paragraph position="0"> In order to experiment the proposed model, we used the tagged corpus shown in section 1. The corpus is divided into the training corpus with 428 dialogues, 8,349 utterances (19.51 utterances per dialogue), and the testing corpus with 100 dialogues, 1,936 utterances (19.36 utterances per dialogue). Using the Maximum Entropy Modeling Toolkit (Ristad 1996), we estimated the model parameter ~ corresponding to each feature functionf in equation (13).</Paragraph>
    <Paragraph position="1"> We made experiments with two models for each analysis model. Modem uses only the unified feature function, and Model-II uses the unified feature function and the separated feature function together. Among the ways to combine the unified feature function with the separated feature function, we choose the combination in which the separated feature function is used only when there is no training example applicable for the unified feature function.</Paragraph>
    <Paragraph position="2"> First, we tested the speech act analysis model and the discourse analysis model. Table 4 and 5 show the results for each analysis model. The results shown in table 4 are obtained by using the correct structural information of discourse, i.e., DSB, as marked in the tagged corpus.</Paragraph>
    <Paragraph position="3"> Similarly those in table 5 are obtained by using the correct speech act information from the tagged corpus.</Paragraph>
    <Paragraph position="4">  In the closed test in table 4, the results of Model-I and Model-II are the same since the probabilities of the unified feature functions always exist in this case. As shown in table 4, the proposed models show better results than previous work, Lee (1997). As shown in table 4 and 5, ModeMI shows better results than Model- null I in all cases. We believe that the separated feature functions are effective for the data sparseness problem. In the open test in table 4, it is difficult to compare the proposed model directly with the previous works like Samuel (1998) and Reithinger (1997) because test data used in those works consists of English dialogues while we use Korean dialogues.</Paragraph>
    <Paragraph position="5"> Furthermore the speech acts used in the experiments are different. We will test our model using the same data with the same speech acts as used in those works in the future work. We tested the integrated dialogue analysis model in which speech act and discourse structure analysis models are integrated. The integrated model uses ModeMI for each analysis model because it showed better performance. In this model, after the system determing the speech act and DSB of an utterance, it uses the results to process the next utterance, recursively. The experimental results are shown in table 6.</Paragraph>
    <Paragraph position="6"> As shown in table 6, the results of the integrated model are worse than the results of each analysis model. For top-1 candidate, the performance of the speech act analysis fell off about 2.89% and that of the discourse structure analysis about 7.07%. Nevertheless, the integrated model still shows better performance than previous work in the speech act analysis.</Paragraph>
  </Section>
class="xml-element"></Paper>