<?xml version="1.0" standalone="yes"?>
<Paper uid="P06-2091">
  <Title>Translating HPSG-style Outputs of a Robust Parser into Typed Dynamic Logic</Title>
  <Section position="6" start_page="710" end_page="713" type="concl">
    <SectionTitle>
4 Experiment
</SectionTitle>
    <Paragraph position="0"> The number of rules we have implemented is shown in Table 1. We used the Penn Treebank (Marcus, 1994) Section 22 (1,527 sentences) to develop and evaluate the proposed method and Section 23 (2,144 sentences) as the final test set.</Paragraph>
    <Paragraph position="1"> We measured the coverage of the construction of TDL semantic representations, in the manner described in a previous study (Bos et al., 2004). Although the best method for strictly evaluating the proposed method is to measure the agreement between the obtained semantic representations and the intuitions of the speaker/writer of the texts, this type of evaluation could not be performed because of insufficient resources. Instead, we measured the rate of successful derivations as an indicator of the coverage of the proposed system. The sentences in the test set were parsed by a robust HPSG parser (Miyao et al., 2005), and HPSG parse trees were successfully generated for 2,122 (98.9%) sentences. The proposed method was then applied to these parse trees. Table 2 shows that 88.3% of the un- null seen sentences are assigned TDL semantic representations. Although this number is slightly less than 92.3%, as reported by Bos et al., (2004), it seems reasonable to say that the proposed method attained a relatively high coverage, given the expressive power of TDL.</Paragraph>
    <Paragraph position="2"> The construction of TDL semantic representations failed for 11.7% of the sentences. We classified the causes of the failure into two types. One of which is application failure of the assignment rules (assignment failure); that is, no assignment rules are applied to a number of HPSG lexical items, and so no TDLESs are assigned to these items. The other is application failure of the composition rules (composition failure). In this case, a type mis-match occurred in the composition, and so a TDLES was not derived.</Paragraph>
    <Paragraph position="3"> Table 3 shows further classification of the causes categorized into the two classes. We manually investigated all of the failures in the development set.</Paragraph>
    <Paragraph position="4"> Assignment failures are caused by three factors. Most assignment failures occurred due to the limitation in the number of the assignment rules (as indicated by &amp;quot;unimplemented words&amp;quot; in the table). In this experiment, we did not implement rules for infrequent HPSG lexical items. We believe that this type of failure will be resolved by increasing the number of  you're in the groove, you see every ball tremendously,&amp;quot; he lectured.</Paragraph>
    <Paragraph position="5"> assignment rules. The second factor in the table, &amp;quot;TDL unsupported words&amp;quot;, refers to expressions that are not covered by the current theory of TDL. In order to resolve this type of failure, the development of TDL is required.</Paragraph>
    <Paragraph position="6"> The third factor, &amp;quot;nonlinguistic HPSG lexical items&amp;quot; includes a small number of cases in which TDLESs are not assigned to the words that are categorized as nonlinguistic syntactic categories by the HPSG parser. This problem is caused by ill-formed outputs of the parser.</Paragraph>
    <Paragraph position="7"> The composition failures can be further classified into three classes according to their causativefactors. Thefirstfactoristheexistence of HPSG schemata for which we have not yet implemented composition rules. These failures will be fixed by extending of the definition of our composition rules. The second factor is type mismatches due to the unintended assignments of TDLESs to lexical items. We need to further elaborate the assignment rules in order to deal with this problem. The third factor is parse trees that are linguistically invalid.</Paragraph>
    <Paragraph position="8"> The error analysis given above indicates that we can further increase the coverage through the improvement of the assignment/composition rules.</Paragraph>
    <Paragraph position="9"> Figure 6 shows an example of the output for a sentence in the development set. The variables $1, ...,$11 are indices that  represent entities, events and situations. For example, $3 represents a situation and $2 represents the lecturing event that exists in $3. past($3) requires that the situation is past. agent($2,$1) requires that the entity $1 is the agent of $2.</Paragraph>
    <Paragraph position="10"> content($2,$4) requires that $4 (as a set of possible worlds) is the content of $2. be($11,$4) refers to $4.Finaly, every($6)[ball($6,$4)][see($7,$4) ...] represents a generalized quantifier &amp;quot;every ball&amp;quot;. The index $6 serves as an antecedent both for bound-variable anaphora within its scope and for E-type anaphora outside its scope. The entities that correspond to the two occurrences of &amp;quot;you&amp;quot; are represented by $8 and $5. Their unification is left as an anaphora resolution task that can be easily solved by existing statistical or rule-based methods, given the structural information of the TDL semantic representation.</Paragraph>
    <Paragraph position="11"> 5Conclusion The present paper proposed a method by which to translate HPSG-style outputs of a robust parser (Miyao et al., 2005) into dynamic semantic representations of TDL (Bekki, 2000). We showed that our implementation achieved high coverage, approximately 90%, for real text of the Penn Treebank corpus and that the resulting representations have sufficient expressive power of contemporary semantic theory involving quantification, plurality, inter/intrasentential anaphora and presupposition. In the present study, we investigated the possibility of achieving robustness and descriptive adequacy of semantics. Although previously thought to have a trade-off relationship, the present study proved that robustness and descriptive adequacy of semantics are not intrinsically incompatible, given the transparency between syntax and discourse semantics.</Paragraph>
    <Paragraph position="12"> If the notion of robustness serves as a criterion not only for the practical usefulness of natural language processing but also for the validity of linguistic theories, then the compositional transparency that penetrates all levels of syntax, sentential semantics, and discourse semantics, beyond the superficial difference between the laws that govern each of the levels, might be reconsidered as an essential principle of linguistic theories.</Paragraph>
  </Section>
class="xml-element"></Paper>