<?xml version="1.0" standalone="yes"?>
<Paper uid="P99-1079">
  <Title>Analysis of Syntax-Based Pronoun Resolution Methods</Title>
  <Section position="4" start_page="0" end_page="602" type="metho">
    <SectionTitle>
2 Left-Right Centering Algorithm
</SectionTitle>
    <Paragraph position="0"> Left-Right Centering (LRC) is a formalized algorithm built upon centering theory's constraints and rules as detailed in Grosz et. al (1995). The creation of the LRC Algorithm is motivated by two drawbacks found in the BFP method. The first is BFP's limitation as a cognitive model since it makes no provision for incremental resolution of pronouns (Kehler 1997). Psycholinguistic research support the claim that listeners process utterances one word at a time, so when they hear a pronoun they will try to resolve it immediately. If new information comes into play which makes the resolution incorrect (such as a violation of binding constraints), the listener will go back and find a correct antecedent. This incremental resolution problem also motivates Strube's S-list approach.</Paragraph>
    <Paragraph position="1"> The second drawback to the BFP algorithm is the computational explosion of generating and filtering anchors. In utterances with two or more pronouns and a Cf-list with several candidate antecedents for each pronoun, thousands of anchors can easily be generated making for a time consuming filtering phase. An exam- null ple from the evaluation corpus illustrates this problem (the italics in Un-1 represent possible antecedents for the pronouns (in italics) of Un): Un-l: Separately, the Federal Energy Regulatory Commission turned down for now a request by Northeast seeking approval of its possible purchase of PS of New Hampshire.</Paragraph>
    <Paragraph position="2"> Un: Northeast said it would refile its request and still hopes for an expedited review by the FERC so that it could complete the purchase by next summer if its bid is the one approved by the bankruptcy court.</Paragraph>
    <Paragraph position="3"> With four pronouns in Un, and eight possible antecedents for each in Un-1, 4096 unique Cf-lists are generated. In the cross-product phase, 9 possible Cb's are crossed with the 4096 Cf's, generating 36864 anchors.</Paragraph>
    <Paragraph position="4"> Given these drawbacks, we propose a revised resolution algorithm that adheres to centering constraints. It works by first searching for an antecedent in the current utterance 2, if one is not found, then the previous Cf-lists (starting with the previous utterance) are searched left-to-right for an antecedent:  1. Preprocessing - from previous utterance: Cb(Un-1) and Cf(Un-1) are available.</Paragraph>
    <Paragraph position="5"> 2. Process Utterance - parse and extract incrementally from Un all references to discourse entities. For each pronoun do: (a) Search for an antecedent intrasententially in Cf-partial(Un) 3 that meets feature and binding constraints.</Paragraph>
    <Paragraph position="6"> If one is found proceed to the next pronoun within utterance. Else go to (b).</Paragraph>
    <Paragraph position="7"> (b) Search for an antecedent intersententially in Cf(Un-1) that meets feature and binding constraints.</Paragraph>
    <Paragraph position="8"> 3. Create Cf- create Cf-list of Un by rank- null ing discourse entities of Un according to grammatical function. Our implementation used a left-right breadth-first walk of the parse tree to approximate sorting by grammatical function.</Paragraph>
    <Paragraph position="9">  resolved, use the criteria from (Brennan et al., 1987) to assign the transition.</Paragraph>
    <Paragraph position="10"> It should be noted that BFP makes use of Centering Rule 2 (Grosz et al., 1995), LRC does not use the transition generated or Rule 2 in steps 4 and 5 since Rule 2's role in pronoun resolution is not yet known (see Kehler 1997 for a critique of its use by BFP).</Paragraph>
    <Paragraph position="11"> Computational overhead is avoided since no anchors or auxiliary data structures need to be produced and filtered.</Paragraph>
  </Section>
  <Section position="5" start_page="602" end_page="602" type="metho">
    <SectionTitle>
3 Evaluation of Algorithms
</SectionTitle>
    <Paragraph position="0"> All four algorithms were run on a 3900 utterance subset of the Penn Treebank annotated corpus (Marcus et al., 1993) provided by Charniak and Ge (1998). The corpus consists of 195 different newspaper articles. Sentences are fully bracketed and have labels that indicate word-class and features. Because the S-list and BFP algorithms do not allow resolution of quoted text, all quoted expressions were removed from the corpus, leaving 1696 pronouns (out of 2026) to be resolved.</Paragraph>
    <Paragraph position="1"> For analysis, the algorithms were broken up into two classes. The &amp;quot;N&amp;quot; group consists of algorithms that search intersententially through all Cf-lists for an antecedent. The &amp;quot;1&amp;quot; group consists of algorithms that can only search for an antecedent in Cf(Un-1). The results for the &amp;quot;N&amp;quot; algorithms and &amp;quot;1&amp;quot; algorithms are depicted in Figures 1 and 2 respectively.</Paragraph>
    <Paragraph position="2"> For comparison, a baseline algorithm was created which simply took the most recent NP (by surface order) that met binding and feature constraints. This naive approach resolved 28.6 percent of pronouns correctly. Clearly, all four perform better than the naive approach. The following section discusses the performance of each algorithm.</Paragraph>
  </Section>
  <Section position="6" start_page="602" end_page="603" type="metho">
    <SectionTitle>
4 Discussion
</SectionTitle>
    <Paragraph position="0"> The surprising result from this evaluation is that the Hobbs algorithm, which uses the least amount of information, actually performs the best. The difference of six more pronouns right  significant so one may conclude that the new centering algorithm is also a viable method. Why do these algorithms perform better than the others? First, both search for referents intrasententially and then intersentially. In this corpus, over 71% of all pronouns have intrasentential referents, so clearly an algorithm that favors the current utterance will perform better. Second, both search their respective data structures in a salience-first manner. Intersententially, both examine previous utterances in the same manner. LRC-N sorts the Cf-list by grammatical function using a breadth-first search and by moving prepended phrases to a less salient position. While Hobbs' algorithm does not do the movement it still searches its parse tree in a breadth-first manner thus emulating the Cf-list search. Intrasententially, Hobbs gets slightly more correct since it first favors antecedents close to the pronoun before searching the rest of the tree. LRC favors entities near the head of the sentence under the assumption they are more salient. The similarities in intra- and intersentential evaluation are reflected in the similarities in their percent right for the respective categories.</Paragraph>
    <Paragraph position="1"> Because the S-list approach incorporates both semantics and syntax in its familiarity ranking scheme, a shallow version which only uses syntax is implemented in this study. Even though several entities were incorrectly labeled, the shallow S-list approach still performed quite well, only 4 percent lower than Hobbs and LRCi. null The standing of the BFP algorithm should not be too surprising given past studies. For example, Strube (1997) had the S-list algorithm performing at 91 percent correct on three New York Times articles while the best version of BFP performed at 81 percent. This ten percent difference is reflected in the present evaluation as well. The main drawback for BFP was its preference for intersentential resolution. Also, BFP as formally defined does not have an intrasentential processing mechanism. For the purposes of the project, the LRC intrasentential technique was used to resolve pronouns that were unable to be resolved by the BFP (intersentential) algorithm.</Paragraph>
    <Paragraph position="2"> In additional experiments, Hobbs and LRC-N were tested with quoted expressions included. LRC used an approach similar to the one proposed by Kamayema (1998) for analyzing quoted expressions. Given this new approach, 70.4% of the 2026 pronouns were resolved correctly by LRC while Hobbs performed at 69.8%, a difference of only 13 pronouns right.</Paragraph>
  </Section>
  <Section position="7" start_page="603" end_page="604" type="metho">
    <SectionTitle>
5 Conclusions
</SectionTitle>
    <Paragraph position="0"> This paper first presented a revised pronoun resolution algorithm that adheres to the constraints of centering theory. It is inspired by the need to remedy a lack of incremental processing and computational issues with the BFP algorithm. Second, the performance of LRC was compared against three other leading pronoun resolution algorithms based solely on syntax. The comparison of these algorithms is  significant in its own right because they have not been previously compared, in computerencoded form, on a common corpus. Coding all the algorithms allows one to quickly test them all on a large corpus and eliminates human error, both shortcomings of hand evaluation.</Paragraph>
    <Paragraph position="1"> Most noteworthy is the performance of Hobbs and LRC. The Hobbs approach reveals that a walk of the parse tree performs just as well as salience based approaches. LRC performs just as well as Hobbs, but the important point is that it can be considered as a replacement for the BFP algorithm not only in terms of performance but in terms of modeling. In terms of implementation, Hobbs is dependent on a precise parse tree for its analysis. If no parse tree is available, Strube's S-list algorithm and LRC prove more useful since grammatical function can be approximated by using surface order.</Paragraph>
  </Section>
class="xml-element"></Paper>