<?xml version="1.0" standalone="yes"?>
<Paper uid="C92-1044">
  <Title>An Acquisition Model for both Choosing and Resolving Anaphora in Conjoined Mandarin Chinese Sentences</Title>
  <Section position="3" start_page="0" end_page="0" type="metho">
    <SectionTitle>
2 Review of previous
</SectionTitle>
    <Paragraph position="0"/>
    <Section position="1" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
approaches
2.1 Search-based approaches
</SectionTitle>
      <Paragraph position="0"> Both history list \[1\] and Hobbs's naive syntactic .algorithm \[7\] are search-bused approaches for re~lvmg anaphora. However, it's not quite obvious to tell which was better than the other with only few exampies. Thus, we collected 120 testing instances to test them. Those instances were selected from linguists' ACTES DE COLING-92, NANTES, 23-28 Aotrr 1992 2 7 4 PROC. OF COLING.92, NANTES, AUO. 23-28, 1992 examples, textbooks, essays and novels. Half of them contained zero anaphora the other pronominal.</Paragraph>
      <Paragraph position="1"> The result showed that the correct number was 111(92.5%) with Hobbs's syntactic algorithm and 87(72.5%) with the history list approach if first matched were selected. There was 109(90.8%) correct for history list if the last matched were selected.</Paragraph>
      <Paragraph position="2"> It seemed that both approaches were applicable to resolve anaphora, tiowever, when there are several NPs with the same semantic features, both approaches may get into troubles. \]~harthermore, both cannot be used to choose anaphora.</Paragraph>
    </Section>
    <Section position="2" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
2.2 LinguisCs criteria
</SectionTitle>
      <Paragraph position="0"> Among linguists' works \[31 \[51 \[lll \[121 \[141, Tai's criteria \[14\] was applicable to both choose and resolve anaphora. Others' suffered from difficulties of extracting features or resolving anaphora. Table 1 shows 4 co-references for Tai's citeria, which are all applicable when co-referred NPs are human. For example, consider the following conjoined sentences: Tai:\[ Lao Zhang \] dao-le Meiguoyihou. \[ \] jiac-le hen-duo pengyou.</Paragraph>
      <Paragraph position="1"> John came U.S.A after \[\] made many friends Since John came to the U.S.A., he has made many friends.</Paragraph>
      <Paragraph position="2"> The subject in the first sentence is human and co-referred by the subject in the second sentence, so this is a subject-subject co-reference. According to Table 1, zero anaphora is preferred to the pronominal one and nominal anaphora is not permitted in this example. Though Tal didn't propose the criteria for resolving anaphora, it was possible to get these criteria just by transforming the choosing criteria in reverse order. After Tai's criteria were applied to choose and resolve anaphora on the 120 testing instances, we got the success numbers 86(71.7%) and 65(54.2%) respectively. The results failed to meet our satisfaction. Through above paragraphs, it appears that search-based methods have their limitations due to lack of enough linguistisc knowledge and Tai's criteria seems to be applicable to both choose and resolve anaphora.</Paragraph>
      <Paragraph position="3"> It might be that Tai's criteria were too general to lead to a high success rate. More reliable method to acquire regularity might be required to promote the success rate. We hypothesized the regularity of anaphora could be accounted by causal relations between the features in the conjoined sentences and the antecedents. In the following section, an acquisition model is introduced.</Paragraph>
    </Section>
  </Section>
  <Section position="4" start_page="0" end_page="0" type="metho">
    <SectionTitle>
3 G-UNIMEM: A Case-Based
</SectionTitle>
    <Paragraph position="0"/>
    <Section position="1" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
Learning Model
</SectionTitle>
      <Paragraph position="0"> In natural language acquisition problem, the restriction of positive-only examples \[2\] has prohibited many machine learning models as a feasible natural language model. However, a case-baaed learning approach such as Lebowitz's UNIMEM \[9\] \[I0\] seems to be a candidate due to its capability to form concepts incrementally from a rich input domain. Nevertheless, to apply UNIMEM directly to the acquisition of anaphoric regularity in Mandarin Chinese is still not sufficient. We have therefore modified UNIMEM into G-UNIMEM.</Paragraph>
      <Paragraph position="1"> G-UNIMEM, a modified version of UNIMEM, is an incremental learning system that uses GBM(Geueralized-based Memory) to generalize concepts from a large set of training instances. The program was implemented in Quintus PROLOG and on SUN workstation.</Paragraph>
      <Paragraph position="2"> G-UNIMEM differs from UNIMEM in two respects.</Paragraph>
      <Paragraph position="3"> Firstly, if a drinker got drunk many times after taking either whiskey and water or brandy and water, he would induce that water made him drunk with UNIMEM. This is intuitively incorrect. Whereas, with G-UNIMEM, he would induce that whiskey and water, brandy and water or water would cause him drunk. In this case, G-UNIMEM retains the possible causal accounts without committing to erroneous conclusion. Secondly, G-UNIMEM can extract explicit causal rules from memory hierarchy.</Paragraph>
      <Paragraph position="4"> Similar to UNIMEM, G-UNIMEM organizes input training instances into a memory hierarchy according to the frequencies of features. However, its goal is to explicitly express the generalized causal relationships between two specified types of features: cause features and goal features. Since there may be inconsistency due to lack of cause features, further refinemeat is needed to obtain consistent causal relations. Thus, there are four different modules in G-UNIMEM to complete different functions in order to achieve this purpose.</Paragraph>
    </Section>
    <Section position="2" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
3.1 The classifier
</SectionTitle>
      <Paragraph position="0"> The classifier is the first module that processes all training instances for G-UNIMEM. Its function is close to UNIMEM that organizes a hierarchy structure to incrementally accommodate a training instance and at the same time generalize the features based on similarities among training instances. The forming hierarchy is organized as either a g-c-hierarchy or a c-g-hierarchy depending on the setup of system, which is defined in Definition 1. In Appendix A we show the basic classifier algorithm.</Paragraph>
      <Paragraph position="1"> Definition 1 A g-c-hierarchy is the hierarchy that every generalized goal feature resides in a GEN-NODE and there is no generalized cause feature that resides between the root node and this GEN-NODE. A c-g-hierarchy doesn't allow any generalized goal feature to reside in the GEN-NODE between the root node and any GEN-NODE where generalized cause features reside. null Figure 1 and Figure 2 show the forming g-c-hierarchy and c-g-hierarchy respectively after 13 annotated training sentences are entered into G-UNIMEM. Generally, g-c-hierarchy would be chosen since it retained all possible causal accounts. For example, the drinker with g-c-hierarchy would induce that whiskey and water, brandy and water or water would cause him drunk; whereas, he would induce whiskey and Ac'rY.s DE COLING-92, NANTES, 23-28 AOU'r 1992 2 7 5 FROC. OI' COLING-92, NAhrrES, AUG. 23-28, 1992 water, brandy and water with e-g-hierarchy. The c-g-hierarchy is more efficient since no rules are needed to be generated. Fig. 3 and Fig. 4 show the updating of a GBM before and after inserting a new training instance.</Paragraph>
    </Section>
    <Section position="3" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
3.2 The rule generator
</SectionTitle>
      <Paragraph position="0"> Once a hierarchy has been constructed by the classifter, the causal rules can be extracted. The rule generator module serves as the role to extract causal rules from the hierarchy. It generates all causal rules from the hierarchy as the regularity is retrieved for predictions. null In Fig. 6, if a testing instance is given for choosing anaphora with a query feature list \[ (g,type(*?)), (g,ante(theme)), (c,fl(theme)), (c,anaphor(theme)), (c,s2(obj)), (c,p(pv))\], the retrieval process is searched with a post-order traverse, namely, in the order sequence of the node number 1, 2, 3, 4, 5 and 6. Since there may be more than one candidate, the system can be setup to select either the first or the most specific one. If the first one is preferred, type(nil) is yielded as the prediction. If the most specific answer is preferred, all possible rules will be tried and the one with the most number of contingent features matched will be the answer(i.e, type(pronoun) ).</Paragraph>
      <Paragraph position="1"> The sample rules generated from Fig. 1 are shown in Fig. 5. Before generating rules, the GBM is adjusted so that all children of a GEN-NODE are ordered according to their confidence scores of features. Then all rules are generated in a post-order traversal.</Paragraph>
    </Section>
    <Section position="4" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
3.3 The rule filter
</SectionTitle>
      <Paragraph position="0"> The rule filter removes those rules that are ill-formed and useless. For example, the causal rule 5 in Fig. 5 has no causes which is not a well-formed rule. It also detects conflicting rules. Conflicting rules are those that have different goal feature descriptions, which are accounted by the same cause. For example, the rule I and rule 6 in Fig. 5 are conflicting. These rules will be detected in this module and then to be resolved by the feature selector.</Paragraph>
    </Section>
    <Section position="5" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
3.4 The feature selector
</SectionTitle>
      <Paragraph position="0"> Any two conflicting rules are resolved by the feature selector through augmenting the two rules with mutual exclusive contingent cause features, which are prepared in advance. Dominant features were used in initial regularity acquisition stage; whereas contin- g ent features were used in feature selection stage. The ominant features such as goal features are assumed to be those that must be present in every anaphoric rule.</Paragraph>
      <Paragraph position="1"> Contingent features are optional. Fig. 6. shows the GBM with g-c hierarchy after feature selection procC/08. null</Paragraph>
    </Section>
  </Section>
  <Section position="5" start_page="0" end_page="0" type="metho">
    <SectionTitle>
4 Tests using sentences anno-
</SectionTitle>
    <Paragraph position="0"> tated with mixed features We trained G-UNIMEM with 30, 60, 90, 120 instances using those features mentioned by Tai, and used all the 120 instances as testing instances. It showed that the approach using Tai's criteria was not promising. There are two reasons. First, none of the success rates was as high as those using the history list approach or IIobbs's algorithm. Second, many conflicting rules remained conflicting due to either that no further features from feature selection were available or too many specific training leading to too many specific rules. These factors decreased the success rate.</Paragraph>
    <Section position="1" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
4.1 Selecting mixed features
</SectionTitle>
      <Paragraph position="0"> Since Tal's features were not sufficient, more semantic features were considered. Among several linguists' works, we tentatively selected some computational feasible syntactic and semantic features from different sources \[3\] \[5\] \[11\] \[12\] \[13\] \[14\] \[15\] as in Table 2. An example with annotated features is shown below. Tile notation \[ \] represents zero anaphora.</Paragraph>
      <Paragraph position="1"> (C)\[Lao zheng\]i qu-le ji-ge \[nurenli.\[ \]j hen hui zuo-cai.</Paragraph>
      <Paragraph position="2"> John married a woman t \] wetlcan cook.</Paragraph>
      <Paragraph position="3"> agent theme agent</Paragraph>
      <Paragraph position="5"> where the notations g and c represent goa and cause features respectively.</Paragraph>
    </Section>
    <Section position="2" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
4.2 Testing using mixed features
</SectionTitle>
      <Paragraph position="0"> After semantic features has been determined, we trained G-UNIMEM with 30, 60, 90, 120 instances and used all the 120 instances as testing instances each time.We hypothesized to choose semantic roles(i.e.</Paragraph>
      <Paragraph position="1"> ease) as dominant cause features. The features such as ante(CASE), type(X), anaphor(CASE) and fi (CASE) are dominant features and the number of fi is variant. The hypothesis was motivated by Sidner \[13\] who used semantic roles to determine focus and resolve definite anaphora. The others such as h(Hm), p(POS), s2(SYN); d(D), con(s) belong to contingent features.</Paragraph>
    </Section>
    <Section position="3" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
4.3 The experimental results
</SectionTitle>
      <Paragraph position="0"> It is interesting that the success numbers in Table 3 increased with the number of training instances.</Paragraph>
      <Paragraph position="1"> Finally, our results showed that experiments with c-g-hierarchy had a little high accuracy rates (95.8% for resolving and 90.8% for choosing anaphora with 120 training instances) than thoee with g-e-hierarchy.</Paragraph>
      <Paragraph position="2"> Both accuracy rates were higher than those with TaPs criteria \[14\]. Thus, G-UNIMEM with semantic roles as dominant features promised much higher accuracy rate.</Paragraph>
      <Paragraph position="3"> In Appendix B we show some sample rules acquired in Horn-like clauses. After examination, either the agent or ~heme of first sentence is most likely to AcrEs DE COLING-92, NANTES, 2.3-28 AOOT 1992 2 7 6 PROC. OF COLING-92. NANTES, AUG. 23-28, 1992 act as antecedents of anaphora. Tiffs phenomenon is in coincidence with the investigation on anaphora by Sidner. That is, the agent often appeared as actor focus and theme as default focus . This is similar to Tai's criteria but is in more compact interpretation.</Paragraph>
    </Section>
  </Section>
  <Section position="6" start_page="0" end_page="0" type="metho">
    <SectionTitle>
5 Discussion
</SectionTitle>
    <Paragraph position="0"> There are two concerns in implementing GUNIMEM: null (1) The feature set : Is the assignment of dominant features and contingent features objective? If there is any contingent feature in the assignment that obvi~ ously improves the accuracy rate, it shonld be assigned as dominant feature. We use statistical methods \[8\] to analyze if contingent features actually improve accuracy rates. If there is no obvious improvement with contingent features, the division of dominant and corrtingent features is acceptable.</Paragraph>
    <Paragraph position="1"> We made the null hypothesis &amp;quot;G-UNIMEM with c-g-hierarchy doesn't have obvious improvement with contingent features&amp;quot; and the alternative hypothesis &amp;quot;G-UNIMEM with c-g-hierarchy has obvious improvement with contingent features&amp;quot;. We titan got two test values from test statistics: tl = 0.8472 and t2 &lt; 0. Both test statistic.q were less than t~ = .05 (= 1.734 with d.f. = 18). Thus, the null hypothesis &amp;quot;G-UNIMEM with c-g-lfierarchy doesn't have obvious improvement with contingent features&amp;quot; was not rejected, which justified that G-UNIMEM using semantic roles as dominant features was valid.</Paragraph>
    <Paragraph position="2"> (2)The sample size : Compared with actual linguistic domain, the 120 training and testing instances are small. A large corpus is desirable to test the system's performance. If it becomes available, our resnlts would be more objective and reliable.</Paragraph>
  </Section>
class="xml-element"></Paper>