File Information
File: 05-lr/acl_arc_1_sum/cleansed_text/xml_by_section/metho/04/j04-2001_metho.xml
Size: 65,194 bytes
Last Modified: 2025-10-06 14:08:45
<?xml version="1.0" standalone="yes"?> <Paper uid="J04-2001"> <Title>c(c) 2004 Association for Computational Linguistics Inferable Centers, Centering Transitions, and the Notion of Coherence</Title> <Section position="5" start_page="121" end_page="124" type="metho"> <SectionTitle> 3. The Corpus </SectionTitle> <Paragraph position="0"/> <Section position="1" start_page="121" end_page="123" type="sub_section"> <SectionTitle> 3.1 The Nature of the Corpus </SectionTitle> <Paragraph position="0"> The data examined consist of a collection of 32 e-mail messages exchanged among five employees of a Japanese company from June 5 to June 16, 1995. The messages constitute a collective attempt to schedule a sports-watching outing convenient for and interesting to all five in the group. Thus, the tone is usually casual. The authors combine standard aspects of written text with various strategies for encoding speech-like information in the messages: nonstandard uses of punctuation, katakana (the syllabary for writing foreign words), and English; nonstandard spelling; emoticons; discourse markers, sentence-final particles, tense, and formality typical of speech; and fillers (Fais 2001; Fais and Yamura-Takei 2003). Quantitative information for the corpus is given in Table 2.</Paragraph> <Paragraph position="1"> Table 2 Number of messages, paragraphs, sentences, clauses, and characters per author.</Paragraph> <Paragraph position="2"> speech and nonreport complements. Our corpus does not contain examples of these kinds of utterances. On the other hand, it does contain tensed clauses acting as &quot;relative clauses&quot; and as verbal complements. We have not separated these from their heads or matrix clauses. This has no effect on the analysis presented in this article, except for the fact that had we separated these clauses, it would have made the problematic situation we describe later even more marked. The status of these clauses vis-`a-vis centering is a topic in need of extensive investigation.</Paragraph> <Paragraph position="3"> watashi wa 18nichi no yakuluto-yokohamasen wa me TOP 18th on Swallows vs. Baystars game TOP &quot;As for me, since I'm really up for the Swallows vs.</Paragraph> <Paragraph position="4"> Baystars game on 18th, that is OK with me.&quot;</Paragraph> <Paragraph position="6"> demo tekondoh ni kyoumishinshin datta U-san ya but taekwondo in keen interest was U-san and</Paragraph> <Paragraph position="8"> interest, taekwondo M-M-san wa koredeiindeshouka M-san TOP I'm wondering if it is ok &quot;But I'm wondering if it is ok for U-san and M-san, who have shown a keen interest in taekwondo.&quot; Clause (1a) has no Cb, since it occurs at the beginning of the message. The Cp of (1a) is Swallows vs. Baystars game, by virtue of the fact that it is topic-marked. The subject of the copular desu in (1b) is omitted; rule 1 implies that the referent for this zero argument is Swallows vs. Baystars game, which is a correct assignment. A similar process resolves the referent for kore 'this' in (1c). Note that the CONTINUE and RETAIN transitions do, in fact, capture the intuition that this segment of the message is coherent and is &quot;about&quot; the Swallows vs. Baystars game.</Paragraph> <Paragraph position="9"> Example (2) is much more typical of the messages in the corpus and is not as well behaved as (1). Notice that all the Cbs in this example are inferable &quot;from the discourse situation&quot; of U i[?]1 . The preponderance of inferable Cbs is typical; out of 330 Cbs in the corpus, 250 (more than 75%) are entities other than pronouns, zero 3 We will ignore first-person arguments in this analysis, since centering is intended to handle only third-person arguments. How topic-marked first-person arguments affect attentional states in discourse is an interesting question, though beyond the scope of this article.</Paragraph> <Paragraph position="10"> We use the following abbreviations for Japanese case markers in this article: TOP, topic; SUBJ, subject; OBJ, object.</Paragraph> <Paragraph position="11"> 4 There is no consensus as to the ordering of the arguments in a Japanese AnoBconstruction (roughly equivalent to possessives; Tetreault 2001; Matsui 1999; but see Fais 2002). We have listed the arguments in the order suggested in Fais (2002); this has little effect on the present discussion.</Paragraph> </Section> <Section position="2" start_page="123" end_page="124" type="sub_section"> <SectionTitle> Fais Transitions and Coherence </SectionTitle> <Paragraph position="0"> arguments, or explicitly realized entities.</Paragraph> <Paragraph position="1"> This raises a question undiscussed in the centering literature: How do we interpret inferable Cbs in the context of assigning transition types? Example (2) illustrates the problems involved: 2a.</Paragraph> <Paragraph position="2"> tokorode enseki nan desuga by the way restaurant is</Paragraph> <Paragraph position="4"> sendagaya kinpen ni sake wo nomeru sendagaya neighborhood in alcohol OBJ can drink</Paragraph> <Paragraph position="6"> neighborhood, alcohol umai sobaya ga aruto miminishimashita good soba shop SUBJ is have heard &quot;By the way, about the restaurant, I've heard there is a good soba shop, which also serves alcohol, around Sendagaya.&quot; &quot;Choosing this restaurant may not be good because it closes early. I-san, do you have information about this restaurant?&quot;</Paragraph> <Paragraph position="8"> The Cb of (2a) is null, since that clause is the first in the discourse segment. Clause (2b) has three centers, listed in the Cf list. All three of these entities are inferable from the discourse context of (2a). We mentioned previously that there is no principled way to determine the list of inferables of an utterance; it is even more difficult, then, to 5 In the subsequent discussion, for the sake of simplicity, we will refer to this group as &quot;explicit&quot; centers, realizing that zero arguments are not precisely &quot;explicit,&quot; but setting them off in this way from inferable centers.</Paragraph> <Paragraph position="9"> determine the order in which inferables should be listed on a Cf list. Therefore, we cannot say which of the Cfs in (2b) is the &quot;highest ranked&quot; and thus the Cb for (2b). Since we cannot determine which is the Cb, we likewise cannot determine whether the Cb of (2b) is the same as its Cp, and so we cannot label the transition at all.</Paragraph> <Paragraph position="10"> The fact that there is only one Cf for (2c) simplifies the problem somewhat. That Cf, which is inferable from (2b), must also be the Cb of (2c), but since we could not ascertain the Cb of (2b), we do not know if the transition from (2b) to (2c) is a CONTINUE or a SMOOTH SHIFT. Again, the presence of only one Cf in (2d) makes matters easier; we are able to label the transition from (2c) to (2d) a SMOOTH SHIFT.</Paragraph> <Paragraph position="11"> We surmise that I-san may not be inferable from the discourse context, though information (about the restaurant) is, and thus the latter becomes the Cb of (2e). Because this Cb is neither the same as the Cb of (2d) nor the same as its own Cp, the transition from (2d) to (2e) is a ROUGH SHIFT.</Paragraph> <Paragraph position="12"> Using the standard centering definitions, supplemented with the notion of NULL transitions to label transitions to utterances that contain no Cb, we hand-tabulated the number of transitions of each type occurring in the corpus (Table 3). The figures for CONTINUE, RETAIN, SMOOTH SHIFT, and ROUGH SHIFT are those for transitions to utterances containing either explicit, pronominal, or zero-argument Cbs. Given the difficulties in accurately labeling transitions to utterances containing inferable Cbs, we grouped these latter together separately from transitions to utterances with explicit, pronominal, or zero-argument Cbs.</Paragraph> <Paragraph position="13"> There are two points of special interest in Table 3. First, of course, is the particularly high number of transitions to utterances containing inferable Cbs and the number of NULL transitions. At over 40%, the utterances involved in transitions to utterances containing inferable Cbs make up a substantial portion of the corpus. Second are the relative proportions of CONTINUE, RETAIN, SMOOTH SHIFT, and ROUGH SHIFT transitions. Note that, considering just these transitions to utterances containing explicitly realized centers, these proportions are roughly what we expect of coherent text: Most of the shifts are CONTINUE, followed by a respectable number of RETAINs, and a very few SHIFTs.</Paragraph> </Section> </Section> <Section position="6" start_page="124" end_page="132" type="metho"> <SectionTitle> 6 Of course, in the absence of any principled way to determine this, this is merely conjecture. In fact, </SectionTitle> <Paragraph position="0"> because I-san is a part of the ongoing e-mail exchange, he could possibly be considered part of the discourse context. The difficulty is, of course, the lack of any rigorous way to determine what the elements in the discourse context are.</Paragraph> <Paragraph position="1"> Fais Transitions and Coherence 4. Lexical Cohesion and Discourse Coherence: A Revision of Standard Transition</Paragraph> <Section position="1" start_page="125" end_page="125" type="sub_section"> <SectionTitle> Types 4.1 Inferable Centers in Centering Theory </SectionTitle> <Paragraph position="0"> We saw in Section 3.2 that the introduction of inferable centers into a centering analysis leads to indeterminate transition identification. We also saw that transitions to utterances with inferable centers make up a large proportion (over 40%) of the transitions in this corpus. It is important, then, if a centering analysis is to represent the nature of coherence in this corpus accurately, that we make principled provisions for the notion captured by inferable centers in the theory.</Paragraph> <Paragraph position="1"> Only rarely are inferable entities actually listed in analyses in the literature, and then usually when their presence is supported by previous explicit mention and by a clear, tight semantic or syntactic relationship between the entities involved, as in (3), taken from (17) in Kameyama (1998): 3a. It is the apparent intention of the Republican Party to campaign on the carcass of what they call Eisenhower Republicanism.</Paragraph> <Paragraph position="3"> 3b. but the heart stopped beating Cb = Republican Party (inferable Possessor of the heart) 3c. and the lifeblood congealed Cb = Republican Party (inferable Possessor of the lifeblood) Note that (3) avoids at least one of the difficulties we encountered in (2); when there is only one center to deal with, as there is in (2c) and (2d) and (3b) and (3c), the choice of Cb is trivial. However, in cases such as (2a) and (2e), or in (3b) if it had been something like but the heart and lifeblood stopped pumping, we cannot determine what the Cb should be. Further, in these cases, it is impossible to identify the appropriate transition to the following utterance; when the Cb of U i[?]1 is undetermined (as it is for (2b)), the transition to U i (in this case, (2c)) is undeterminable.</Paragraph> </Section> <Section position="2" start_page="125" end_page="126" type="sub_section"> <SectionTitle> 4.2 Logical Difficulties with Inferable Centers </SectionTitle> <Paragraph position="0"> There are further problems resulting from taking the use of inferable centers to its logical extreme. In our analysis so far, we have been concerned with explicit entities in U i that realize centers in U i[?]1 that are inferable from the discourse context. To be accurate, those inferable centers need to be listed in the Cf list for U i[?]1 . However, if we process discourse incrementally, this leads to the conclusion that since we do not know which inferable entity from U i[?]1 will be evoked in U i , we need to list every inferable entity in the Cf list of U i[?]1 . This is both computationally untenable and, in view of the lack of any parameters for determining what constitutes an allowable inferable Cb, impossible. Even if it were possible and desirable, how could we define for inferable entities the type of grammatical-role information essential to determining the placement of these entities on a Cf list? Again taking the definition of inferable centers to its extreme, we note another problem. Every utterance, not just U i[?]1 , evokes inferable centers. There is nothing in the theory to preclude a situation such as that shown in (4) (a version of (2) simplified for illustrative purposes) (implicit inferable centers are given in italics): Computational Linguistics Volume 30, Number 2</Paragraph> <Paragraph position="2"> umai sobaya ga aruto miminishimashita good soba shop SUBJ is have heard &quot;I've heard there is a good soba shop around Sendagaya.&quot; udon shop, sake shop, shop clerk, menu, food, customers...</Paragraph> <Paragraph position="3"> 4b.</Paragraph> <Paragraph position="4"> demo heiten jikoku ga hayaisounandesu but shop closing time SUBJ is early &quot;But the shop closes early.&quot; Cb = shop clerk Cf = shop closing time, shop opening time, shop, shop clerk, door...</Paragraph> <Paragraph position="5"> In (4), an inferable center from the Cf list of (4b) matches an inferable center from the Cf list of (4a) and is chosen as the Cb. Of course, there could be numerous identical inferable centers on the Cf lists of U</Paragraph> <Paragraph position="7"> this is obviously an absurd extension of the inclusion of inferable centers in centering theory, there is nothing, unfortunately, in the theory itself to rule it out.</Paragraph> </Section> <Section position="3" start_page="126" end_page="127" type="sub_section"> <SectionTitle> 4.3 Possible Solutions to the Problems of Inferable Centers </SectionTitle> <Paragraph position="0"> ing references (Clark 1977); they have a conceptual relationship to entities in a previous utterance. There is a sense that bridging references should participate in the creation of coherence in a discourse (Hahn, Markert, and Strube 1996). But the work on bridging references characterizes this relationship as referential or anaphoric; this can be seen in the various terms under which this phenomenon is discussed: bridging references, indirect anaphora, functional anaphora, and partial anaphora. Bridging references, however, unlike the usual case of anaphora, may be mediated not only by a strict identity condition, but also by any number of other semantic relationships (is-a, has-a, made-of, at-time, etc.).</Paragraph> <Paragraph position="1"> Unfortunately, the establishment of the semantic relationship between an anchor and its bridging reference is notoriously difficult. Poesio et al. (2000), even after severely restricting the types of relationships to be labeled, had extremely poor inter-labeler reliability on a first pass. Every account in which bridging references are addressed restricts allowable relationships to a small, relatively well-defined set (Vieira and Poesio 2001; Poesio et al. 2000; Murata, Isahara, and Nagao 1999; Strube and Hahn 1999). Cote (1998) proposes the use of lexical-conceptual primitives instead of grammatical relations in Cf templates and suggests that the conceptual information that this approach provides might be rich enough to supply part-whole information necessary to the resolution of bridging references. She points out as well, however, that a number of other types of semantic relationships manifested in bridging references would not be identifiable from lexical-conceptual information. Thus, although work on bridging references has attempted to provide a characterization of the possible semantic relationships involved, what success has been achieved is limited to a small subset of cases.</Paragraph> </Section> <Section position="4" start_page="127" end_page="128" type="sub_section"> <SectionTitle> Fais Transitions and Coherence </SectionTitle> <Paragraph position="0"> logical problems introduced by inferable centers is to take into account only inferable centers that are explicitly realized in an utterance when determining the Cb of that utterance. This would mean, for example, considering soba shop, Sendagaya neighborhood, and alcohol from (2b) in choosing the Cb for (2c). But how do we make the choice among these possibilities? Any of these three different, explicit, inferable centers could be chosen to be the Cb. But this is exactly the difficulty: We have no principled way to make such a choice. We have no way of knowing which of these inferable centers is ranked highest in the Cf list for (2a) so that we can select that Cf to be the Cb of (2b).</Paragraph> <Paragraph position="1"> A second possibility is to allow only explicitly realized (i.e., noninferable) centers. This seems to be the approach taken by Passonneau (1998); her definition of a null Cb seems to imply that a Cb must be (noninferable and) explicitly realized, and her null Cbs constitute the cases in which there is no explicit Cb. In her examination of the Pear Stories (recordings of people describing to another person a movie they had seen; Chafe 1980), NULL transitions (transitions to an utterance with a null Cb) represent the majority of transitions. Although her concern is discourse segmentation, Passonneau does note that the patterning of transition types does not accurately reflect the coherence of the stories.</Paragraph> <Paragraph position="2"> Allowing only explicit centers would mean, for the corpus studied in this article, that inferable Cbs become null Cbs and the proportion of NULL transitions becomes 75.7%. Under this assumption, this corpus would be characterized as extremely incoherent, a claim belied both by native-speaker intuitions (an acceptable level of coherency for these texts was confirmed by three native speakers) and also by the fact that the task that was the central concern of these messages was successfully completed; the group exchanged a number of opinions and pieces of information and came to a consensus regarding their sports outing, with no message showing confusion about information contained in previous messages. Thus, the solution of allowing only explicit centers does not yield an accurate characterization of the coherence of this corpus.</Paragraph> <Paragraph position="3"> Hurewitz (1998) chose to define allowable inferable Cbs fairly narrowly in her English data, requiring functional dependency or a poset relationship to hold in order for a Cb to be recognized. Even with this definition, which is more constrained than we have taken &quot;inferable&quot; to be in our previous discussion, she finds that 21% of the Brown corpus (a variety of written texts) and 28% of the switchboard corpus (taped telephone conversations) consist of what she calls a no-Cb condition. Poesio et al. (2000) report a similarly high proportion of nonexplicit Cbs in their English text corpora. They test a number of configurations of parameters of centering theory to attempt to minimize the number of null Cbs (reasoning that the best configuration of parameters would result in the fewest violations of the constraints of the theory, in this case, the constraint that all utterances in the discourse except for the first have at least one Cb). One way in which they are able to improve their results significantly is by allowing a restricted set of three types of nonidentity relationships between centers, that is, by recognizing three types of well-defined inferables. However, simply limiting the type of inferables allowed still does not address the issue of the indeterminacy of transitions to utterances containing inferable Cbs. And the central question raised by the high number of nonexplicit Cbs found in naturally occurring texts remains unaddressed: How can we characterize coherence in a text in which Cbs are so often inferable and thus in which transition types are often indeterminate? The crux of the problem lies in the application of standard centering processes to inferable centers. In a nonproblematic case, a Cb in U i is recognized by virtue of its Computational Linguistics Volume 30, Number 2 identity to a Cf in U i[?]1 . This is (relatively) straightforward in the case of explicit centers.</Paragraph> <Paragraph position="4"> But now apply this process to inferable centers. In a standard centering approach, a</Paragraph> <Paragraph position="6"> This implies that we must somehow make available all the possible inferable centers in U i[?]1 in order to recognize (possibly) one of them as the Cb for U i . We have already noted that this position is untenable. Even if we recognize the inferable centers of</Paragraph> <Paragraph position="8"> a posteriori by considering only those that appear in U</Paragraph> <Paragraph position="10"> . What we need, instead, is a way to recognize a Cb in U i not by virtue of its identity with a preestablished list of (explicit and inferable) centers, but by virtue of a relationship, other than identity, with the explicit centers of U i[?]1 . We propose the relationship of lexical cohesion to fill this function. The recognition of a lexically cohesive relationship, then, admits inferable centers without allowing the virtually uncontrollable proliferation of hard-to-define inferable centers in the Cf lists for utterances. We propose a principled way to define this relationship that not only avoids the problems discussed above but also more accurately characterizes the coherence of this corpus.</Paragraph> </Section> <Section position="5" start_page="128" end_page="129" type="sub_section"> <SectionTitle> 4.4 Coherence and Cohesion </SectionTitle> <Paragraph position="0"> Halliday (1994) characterizes cohesion in text as the establishment of &quot;relations within the text that are not subject to [grammatical] limitations; relations that may involve elements of any extent, both smaller and larger than clauses, from single words to lengthy passages of text; and that may hold across gaps of any extent...without regard to the nature of whatever intervenes&quot; (page 309).</Paragraph> <Paragraph position="1"> Cohesion is that aspect &quot;whereby the flow of meaning is channelled into a traceable current of discourse instead of spilling out formlessly in every possible direction&quot; (page 311). It is this &quot;traceable current of discourse&quot; that centering is meant to model.</Paragraph> <Paragraph position="2"> Lexical cohesion contributes to textual coherence. In other words, strong semantic and structural relationships among words in a text help to make that piece of text &quot;make sense.&quot; Coherence is a property of discourse; cohesion is a property of discourse elements. Centering models coherence by characterizing relationships between elements of discourse. We claim that it is not only the continuation of identical explicit discourse elements that creates coherence, but also strong cohesion among discourse elements.</Paragraph> <Paragraph position="3"> Halliday identifies four features of text that create cohesion among discourse elements: conjunction, reference, ellipsis, and lexical cohesion. Insights concerning conjunction types and their interactions with the processes of referent resolution have been elaborated in a number of works (Nariyama 2000; Nakaiwa and Shirai 1996; Kuno 1973) but have not been well integrated into the centering approach. Reference and ellipsis are, of course, some of the mainstays of centering research. Lexical cohesion, however, is an aspect of text coherence that has had only a trivial application in a centering approach, although it has been incorporated into other aspects of natural language processing (see subsequent discussion). &quot;Lexical cohesion,&quot; according to Halliday, &quot;comes about through the selection of items that are related in some way to those that have gone before&quot; (page 330). In centering, that relationship has been 7 We follow Halliday in assuming that &quot;[f]or a text to be coherent, it must be cohesive; but it must be more besides.&quot; He characterizes the &quot;more&quot; as being socially, semantically, and structurally appropriate. We will not deal with these elements here but rather will limit ourselves to the contribution of lexical cohesion to coherence.</Paragraph> <Paragraph position="4"> 8 Of course it is possible to have cohesion without coherence and vice versa; Morris and Hirst (1991) give some nice examples. However, as they assert, &quot;most sentences that relate coherently do exhibit cohesion as well&quot; (page 26).</Paragraph> <Paragraph position="6"> is required to be the same as an element in U i[?]1 (or in its discourse context) in order for it to be considered a Cb. However, earlier we saw the difficulties of admitting inferables and the inability of a centering approach to characterize coherence for inferable centers. Applying the notion of lexical relatedness to cases involving inferables allows us to capture what seems intuitively to constitute the relationship between clauses containing inferable centers.</Paragraph> <Paragraph position="7"> A number of other accounts provide relevant information concerning lexical cohesion in text. These accounts are based upon the characterization of semantic relations among discourse elements by reference to semantic information contained in WordNet (Harabagiu 1998, 1999), thesauruses (Harabagiu 1998; Morris and Hirst 1991; Okumura and Honda 1994), or dictionaries (Kasahara et al. 1996; Kozima 1993; Kozima and Furugori 1993). In all of these approaches, the semantic distance or similarity between (or among) words is computed, and in most of these accounts, the results are applied to the segmentation of discourse.</Paragraph> <Paragraph position="8"> The intuition behind the importance of lexical relatedness has been applied to a number of other tasks in natural language processing and analysis as well. Lotfipour-Saedi (1997) uses lexical cohesion to develop a rigorous notion of &quot;translation equivalence&quot;; Boguraev and Neff (2000) to improve document summarization techniques; Sack (1999) to create &quot;diagrams of social cohesion&quot; for newsgroup postings; and Okumura and Honda (1994) to disambiguate word senses.</Paragraph> <Paragraph position="9"> Halliday asserts that &quot;this interaction between lexical cohesion and reference...is the principal means for tracking a participant through the discourse&quot; (page 332), that is, for modeling focus. Centering has provided us with a principled way of characterizing the tracking of reference; the addition of the notion of lexical cohesion allows centering to function in an even more empirically comprehensive way, by making possible the principled inclusion of what have been called inferable centers. In the next section, we outline how lexical cohesion can be incorporated into a centering theory.</Paragraph> </Section> <Section position="6" start_page="129" end_page="132" type="sub_section"> <SectionTitle> 4.5 COHESIVE Transition and COMPLETE SHIFT </SectionTitle> <Paragraph position="0"> With the use of the sorts of techniques to establish semantic distance described in the works cited earlier, it is possible to be precise about the notion of lexical cohesion.</Paragraph> <Paragraph position="1"> In this section, we discuss how semantic distance is established using one of these techniques, a semantic similarity measure derived from the Gainen Base ('Concept Database') (Kasahara et al. 1996). We then indicate how semantic distance can be used to define the notion of lexical cohesion as a crucial element in the creation of two new types of transition: COHESIVE and COMPLETE SHIFT, which allow us to adequately characterize coherence in a corpus containing a high proportion of nonexplicit Cbs.</Paragraph> <Paragraph position="2"> The Gainen Base is a knowledge base built from machine-readable dictionaries of Japanese. Each word in the knowledge base is defined by a list of weighted keywords extracted from the dictionary definition of the word. The number of times a keyword appears in the word's definitions determines the weight for the keyword. Keywords are standardized to take into account the presence of semantically similar words in the definitions, and their weights are normalized to take into account the differing lengths of definitions in the dictionaries. The semantic distance between two words is calculated as a function of the nearness of the two words in a vector space.</Paragraph> <Paragraph position="4"> our present purposes, then, we say that there is lexical cohesion between U</Paragraph> <Paragraph position="6"> ) as determined using a well-defined semantic similarity measure over the Gainen Base. We will examine in more detail in Section 4.6 whether semantic similarity is best viewed as holding between the sets of Cfs of utterances or between individual discourse entities in those utterances. For now, we will talk equally of lexical cohesion between utterances and lexical cohesion among the Cfs that participate in defining cohesion for those utterances. We define the relation [?] as indicating strong lexical cohesion, with a lexical cohesion factor of one indicating identity.</Paragraph> <Paragraph position="7"> We supplement the standard table of transition states, shown in the left side of Table 4, with transitions defined in the right side. The sense of this table is as follows. We assume all centers to be explicit, that is, pronouns, zero arguments, or explicitly realized entities. The left portion of Table 4 allows us to model transition states in well-behaved explicit contexts, tracking the focus of the discourse in a specific, local way. It includes the cases in which the Cb of U might be &quot;?,&quot; that is, that U i[?]1 might be the discourse-initial utterance or might simply have no Cb, while U i does have (an explicit) Cb. However, where there is not strict identity between any (explicit) element</Paragraph> <Paragraph position="9"> propose a new interpretation for these cases, described in the right portion of Table 4.</Paragraph> <Paragraph position="10"> Table 4 defines two new types of shift to utterances that do not contain an explicit Cb. If there is at least one Cf in U i that has a high lexical cohesion value with some</Paragraph> <Paragraph position="12"> is a COHESIVE transition. This situation is illustrated in clauses (2b), (2c), (2d), and (2e) of example (2). The transitions to these clauses, under the present proposal, are reanalyzed as COHESIVE, as shown in a reanalyzed version of example (2) (entities claimed to bear close semantic relationships to one another are shown in boldface here and in subsequent examples): the following conditions: We refer the reader to Kasahara et al. (1996) for a full discussion of the algorithms used to weight and normalize keywords and to calculate semantic distance.</Paragraph> <Paragraph position="13"> umai sobaya ga aruto miminishimashita good soba shop SUBJ is have heard &quot;By the way, about the restaurant, I've heard there is a good soba shop, which also serves alcohol, around Sendagaya.&quot; COHESIVE 2c.</Paragraph> <Paragraph position="14"> heiten jikoku ga hayaisounanode shop closing time SUBJ early seems because &quot;Choosing this restaurant may not be good because it closes early. I-san, do you know about this restaurant?&quot; Each COHESIVE transition in the modified version of (2) is justified by the presence of a strong semantic relation between at least one Cf in U</Paragraph> <Paragraph position="16"> and at least one Cf in the previous utterance. For example, soba shop in (2b) is semantically related to restaurant in (2a), as is alcohol, probably to a lesser extent, and Sendagaya (if our database includes the information that this is the name of a restaurant). Note that the transition from (2d) to (2e) in this interpretation is a COHESIVE one, rather than a ROUGH SHIFT. Certainly the Cp changes from restaurant to I-san, but since restaurant is still present in the Cf list for (2e), the transition is by no means as abrupt as the designation ROUGH SHIFT implies.</Paragraph> <Paragraph position="17"> The presence of a null Cb in and of itself, then, is not necessarily indicative of incoherence. The level of coherence is captured instead by the proportions of the various transition states present in a corpus, including COHESIVE transitions.</Paragraph> <Paragraph position="18"> Not all utterances containing null Cbs are felt to be cohesive with previous utterances, of course. If there is no explicit Cb and no Cf in U</Paragraph> <Paragraph position="20"> such that it has strong lexical cohesion with a Cf in U i[?]1 , then the shift is considered COMPLETE. This is illustrated in example (5), which shows the continuation of example (1): Computational Linguistics Volume 30, Number 2 5a. Udemo tekondoh ni kyoumishinshin datta U-san ya but taekwondo in keen interest was U-san and</Paragraph> <Paragraph position="22"> youde kyou mo shikkari ame ga futteimasu with today also heavily rain SUBJ is raining &quot;With the declaration that the rainy season has come, it is raining heavily today.&quot;</Paragraph> </Section> </Section> <Section position="7" start_page="132" end_page="143" type="metho"> <SectionTitle> RETAIN </SectionTitle> <Paragraph position="0"/> <Paragraph position="2"> probability, rain There is very low cohesion between the Cfs of (5b), that is, today, rain, declaration, and beginning of rainy season, and those of (5a), namely, U-san, M-san, game, taekwondo, and interest. Thus, we designate the transition from (5a) to (5b) as a COMPLETE SHIFT, which matches our intuition that, in fact, the topic of the message has changed. This is further corroborated by the fact that (5c) RETAINs the Cp of (5b) as its Cb; in other words, it goes on to develop the new topic begun in (5b).</Paragraph> <Paragraph position="3"> Since the two sides of Table 4 are complementary, it is perfectly possible for coherence to be characterized by various combinations of the transitions in the table. Example (6) illustrates the interaction between the two sides of Table 4: 6a.</Paragraph> <Paragraph position="4"> konouchi jinguu no hou wa 17 nichi to 18 nichi of these Jingu TOP 17th and 18th</Paragraph> <Section position="1" start_page="133" end_page="134" type="sub_section"> <SectionTitle> Fais Transitions and Coherence </SectionTitle> <Paragraph position="0"> no yokohamasen ga iikato omotteimasu ga game versus Yokohama SUBJ is good suppose but game, 17th, 18th &quot;Of these, for Jingu Stadium I suppose the game versus Yokohama on the 17th or the 18th would be good. As for J-league, since I don't have further information such as the match-ups, please let me know if you have any information.&quot; &quot;Since there are no good boxing match-ups in this month, I would like to skip it this time.&quot; The transition from (6a) to (6b) is COHESIVE, since (6b) has no Cb and yet the Cfs of the two utterances are semantically related. (6c) RETAINs match-up information from (6b), which is then COHESIVE with match-ups and boxing in (6d). Since (6e) does in fact have an explicit Cb identical both to the Cb of (6d) and to its own Cp, the transition to (6e) is CONTINUE. This latter transition is an example of a non-discourse-initial utterance 11 It is a departure from the standard Cf template for Japanese to designate these as the Cb instead of the topic-marked Jingu Stadium; however, this pronominal form comes immediately after the mention of two options in the previous clause and so seems to be the Cb regardless of its lack of marking. This choice of Cb has no bearing on the point of this example.</Paragraph> <Paragraph position="2"> ) negationslash= ?. (Note that (5) also demonstrates the interaction between the two sides of Table 4, with a RETAIN transition following a COMPLETE SHIFT.) The proposal to include COHESIVE and COMPLETE SHIFT in centering theory is motivated by concerns about having to specify two identical entities in adjacent utterances in order for those utterances to exhibit coherence. The requirement of identity leads to the need to allow inferable entities to play a part. However, it is often impossible to characterize transition states to utterances containing inferable Cbs. By including the notion of COHESIVE transitions, we capture the relatedness of two entities without the need to invoke inferable centers, and we can far better characterize the apparent coherence in this corpus.</Paragraph> <Paragraph position="3"> and COMPLETE SHIFTs, we reinterpreted the labeling of transitions in the corpus (as represented in Table 3) and hand-tabulated a revised distribution of transition types (Table 5). The decision to designate transitions to inferable Cbs and NULL transitions as COHESIVE or COMPLETE SHIFT was made on the basis of an intuitive assessment of the possible semantic relations between entities in adjacent utterances. However, a possible method for automatic determination is described in the next section.</Paragraph> </Section> <Section position="2" start_page="134" end_page="135" type="sub_section"> <SectionTitle> 4.6 A Preliminary Implementation </SectionTitle> <Paragraph position="0"> COHESIVE and COMPLETE SHIFT, we subjected the corpus to an analysis of semantic distance using the Gainen Base. Before implementing the analysis, we processed the corpus so that it contained only the canonical forms of the words in each utterance, the forms accepted by the Gainen Base algorithms. The first step in this procedure was to run the corpus through a morphological analyzer, ALT-JAWS (Nippon Telegraph and Telephone Corporation 1996), which rendered all word forms in their canonical (usually kanji [Chinese character]) form. This is a necessary and standard preprocessing step for most computational analyses of Japanese text, which contains no spaces between word forms to indicate their morphological structure. In addition, writers of e-mail may use hiragana or katakana (the two syllabaries used for writing primarily function words and foreign words, respectively) even for words that have standard kanji forms (Fais and Yamura-Takei 2003). These words need to be rendered in kanji as well. The results of this analysis were checked and corrected by native speakers of Japanese in order to eliminate cases in which the analyzer chose the incorrect kanji form for a homophonous hiragana representation.</Paragraph> <Paragraph position="1"> Table 5 Reanalysis of transitions occurring in the corpus, by type.</Paragraph> </Section> <Section position="3" start_page="135" end_page="135" type="sub_section"> <SectionTitle> Fais Transitions and Coherence </SectionTitle> <Paragraph position="0"> The next step was to delete all but noun forms from the corpus. This step substantiates the notion that the central factors in the centering approach are the discourse entities of utterances (we will return to this in Section 5.3). Further, the antecedents of all pronominal and zero-argument references were made explicit in the text.</Paragraph> </Section> <Section position="4" start_page="135" end_page="140" type="sub_section"> <SectionTitle> This </SectionTitle> <Paragraph position="0"> was done by inserting the kanji forms for these antecedents, as they appeared in the text, into the utterance in which the pronominal or zero-argument form appeared. Antecedents were determined by a native speaker of Japanese. The same native speaker assisted in dividing the text into clauses. Both of these steps, identifying antecedents and parsing sentences into clauses, can be completed, in theory, by automatic, computational methods (Huls, Bos, and Claassen 1995; Nakaiwa and Shirai 1996; Paul and Sumita 2001; Yamura-Takei et al. 2002), but the success rate of these approaches is not high enough to rely on them for completely accurate analyses of this type of corpus at this time. Because our intent is to examine the effectiveness of the use of the Gainen Base in determining lexical cohesion, and not to implement a fully automatic process, we did not attempt to use entirely automatic methods in the preprocessing of the text.</Paragraph> <Paragraph position="1"> erage. Of the 670 types of nouns present in the e-mail corpus, only 235 are found in the machine-readable dictionaries with which the Gainen Base was constructed. What makes this ratio even more problematic is that a number of these missing words (e.g., supo-tsu kansen, 'sports-watching event,' rakurosu, 'lacrosse,' and the names of sports teams) are high-frequency words in this corpus.</Paragraph> <Paragraph position="2"> The problem of coverage in automatic language-processing systems is a common one (Hutchins 1995; Sag et al. 2002; Fujita and Bond 2002). At the level of coverage provided by the Gainen Base, however, we cannot usefully assess how well semantic similarity characterizes coherence in this corpus as a whole. However, we can make this assessment for those clauses in which every noun can be found in the Gainen Base.</Paragraph> <Paragraph position="3"> In order to get an accurate assessment of lexical cohesion between adjacent clauses that fall into this subset of the corpus, we included those cases in which all the nouns in the clause itself as well as those in the clause before it were found the Gainen Base. Out of an original 443 clauses, there are 66 clauses that meet these two criteria.</Paragraph> <Paragraph position="4"> We measured semantic similarity between adjacent clauses in two ways. In the first, we measured the semantic distance between the group of nouns in U i[?]1 and the group of nouns in U</Paragraph> <Paragraph position="6"> . In the second, we measured the semantic distances between each individual noun in U</Paragraph> <Paragraph position="8"> The first method is far &quot;lighter&quot; computationally, but the second method gives us useful information about the contribution of each noun in U</Paragraph> <Paragraph position="10"> to the lexical cohesion between utterances. We will compare the information derived from these two approaches hereafter. 4.6.3 Evaluation of the Use of Semantic Similarity. We assessed in three ways how well semantic similarity can define COHESIVE transitions and thus contribute to the characterization of coherence in the text. First, in the 66 clauses with full coverage by the Gainen Base, we examined the 18 instances of what we had, on the basis of intuitive human judgment, designated COHESIVE transitions. In particular, we focused 12 We did not, however, include the entities involved in event deixis; the identity of these entities is much harder to determine, both for human judges and for automatic language-processing systems.</Paragraph> <Paragraph position="11"> 13 We actually measured semantic similarity in a third way as well, that is, between every possible combination of the individual nouns in U</Paragraph> <Paragraph position="13"> . This yielded the same results as the individual-to-group method reported on here and, of course, is much &quot;heavier&quot; computationally, so we have restricted our discussion to the first two methods.</Paragraph> <Paragraph position="14"> Computational Linguistics Volume 30, Number 2 on the noun(s) we had singled out in our revised hand-tabulation of the corpus (Table 5) as providing the lexical cohesion in these transitions. Using results from the individual measures of semantic similarity (the second method described earlier), we determined the noun in U</Paragraph> <Paragraph position="16"> . We then compared the nouns picked out by human judgment as providing lexical cohesion with those determined computationally to see if they matched. We eliminated three cases in which there was only one noun in U</Paragraph> <Paragraph position="18"> and thus only one possible choice for the lexically cohering entity, which would, by default, have had the highest semantic similarity to the preceding utterance. Out of the 15 remaining examples of COHESIVE transitions, in 13 cases, or 87% of the time, the human judgments and the computationally determined choices matched.</Paragraph> <Paragraph position="19"> The other two assessments of the results are based on centering claims for the relative ease of processing of the different transitions, claims captured in rule 2: CONTINUE transitions impose the lowest inferential load on processors, ROUGH SHIFTs the highest. Since COHESIVE transitions act like CONTINUE transitions but replace the identity condition on Cbs with a similarity condition, we conjecture that they place only a slightly higher load on processing than CONTINUE transitions. Likewise, since COMPLETE SHIFTs represent an even greater discontinuity than ROUGH SHIFTs (there being not only no Cb in the utterance, but also no entity even similar to entities in the previous utterance), we conjecture that COMPLETE SHIFTs impose a higher processing load than ROUGH SHIFTs.</Paragraph> <Paragraph position="20"> We reason that greater semantic similarity corresponds to a lower processing load.</Paragraph> <Paragraph position="21"> Granted this assumption, then, utterances that are connected by CONTINUE transitions have the highest semantic similarity, since CONTINUE represents the lowest processing load; those connected by COMPLETE SHIFTs, the lowest semantic similarity, since COMPLETE SHIFT has the highest processing load. Although the particular ranking of RETAIN, SMOOTH, ROUGH, and COHESIVE transitions is problematic, we are saved from having to make an exact determination by the fact that, among the 66 clauses with full coverage, only CONTINUE, COHESIVE, and COMPLETE SHIFT transitions are well-represented (we simply note the results for the two RETAIN transitions, since this is hardly a large enough sample to be meaningful). Our reasoning then predicts that CONTINUE transitions have the highest semantic similarity measures, followed by COHESIVE SHIFTs, followed by COMPLETE SHIFTs.</Paragraph> <Paragraph position="22"> In our first assessment of this prediction, we averaged semantic measures for each transition type over the 66 clauses with full coverage by the Gainen Base. Table 6 gives the averages obtained for both the groupwise and individual analyses. These results support the ranking of transitions for processing load predicted on the basis of similarity measures. However, averaging over all messages provides only a gross approximation of the values for each transition type. To get a more detailed look at this claim, we examined the semantic distances between entities involved in each type of transition within messages.</Paragraph> <Paragraph position="23"> The 66 clauses with full coverage include six messages with only one clause and four messages with only one transition type represented; this resulted in the elimina14 Where there was more than one noun, as, for example in (2b), we chose the one that seemed, intuitively, to have the strongest semantic connection to a center in the previous utterance. either of these cases), leaving us with a total of 47 clauses grouped into 12 messages. We sorted the types of transitions to these clauses within each message by similarity measure. For each transition, we ascertained whether it fulfilled the prediction for ranking transitions by inferential load that was made on the basis of our earlier assumptions concerning semantic distance. We assigned two scores for each transition: whether it was appropriately positioned with respect to the preceding transition and with respect to the following transition in the sort. Those transitions with the highest and the lowest semantic measures were scored only with respect to the following and the preceding transitions, respectively, and thus received just one score. Two consecutive identical transitions were scored &quot;correct.&quot; Table 7 gives an example of how this scoring was performed for the clauses from one representative message that have full coverage in the Gainen Base. It lists the type of transition to each clause, the semantic distance to the previous clause, and the scores designating whether that transition is appropriately positioned with respect to the previous and following transitions. (Recall that the predicted order is CONTINUE, COHESIVE, COMPLETE.) So, for example, the CONTINUE transition to clause 235 fulfills the ranking prediction with respect to both the previous and the following clauses; its semantic measure is lower than that of only one other clause, which is also a CONTINUE, and is higher than that of a COHESIVE transition. The COMPLETE SHIFT to clause 227, on the other hand, fulfills the prediction vis-`a-vis the previous transition on the list a Since we make no claim as to whether the placement of RETAIN is correct or not, we do not count it in these totals.</Paragraph> <Paragraph position="24"> (i.e., its semantic measure is lower than that of a COHESIVE transition) but violates the prediction with respect to the following transition (i.e., its semantic measure is higher than that of another COHESIVE transition). For 47 clauses over 12 messages, then, the total number of such scores was 70: 24 scores for the transitions having the highest and lowest measures in each message, and two each for the remaining 23 transitions, one for their positions relative to the transitions above, and the other for their positions relative to the transitions below them.</Paragraph> <Paragraph position="25"> Recall that we determined similarity in two different ways: first, for the group of nouns in U i[?]1 and the group of nouns in U</Paragraph> <Paragraph position="27"> Table 8 reports the total number and percentage of correct and incorrect scores for each transition type in both the groupwise and the individual analyses (the measure taken for U</Paragraph> <Paragraph position="29"> in the latter case is the maximum similarity measure out of the measures for all the nouns in U</Paragraph> <Paragraph position="31"> ). The results in this table suggest that similarity scores can accurately represent relative coherence as characterized by the transitions in this small sample. That is, the similarity scores we have examined here reflect the relative load on processing imposed by each type of transition with between 86% and 100% accuracy, with groupwise scores being slightly more accurate than those based on the highest individual score. In addition, similarity measures never predict a CONTINUE transition with a higher processing load (i.e., lower similarity score) than a COMPLETE SHIFT. That is, the relative positions of CONTINUE and COMPLETE SHIFT are always correct.</Paragraph> <Paragraph position="32"> We examined the cases in which similarity scores make an incorrect prediction about relative placement in the scale of processing load. The one incorrect prediction concerning a CONTINUE (in the individual-analysis method) involves an example in</Paragraph> <Paragraph position="34"> in this case (the average number of nouns per clause in the corpus is 2.3). The remaining incorrect predictions involve the assignment of lower similarity scores to COHESIVE transitions than to COMPLETE SHIFTs. Some of these lower scores for COHESIVE transitions are the result of the fact that world knowledge is necessary to infer a connection between the two clauses. This is the case, for example, in (7): enkai taimu niwa choudo iidesuyone drinking party time for TOP exact right</Paragraph> <Paragraph position="36"> &quot;Even if it's far we can come back to Tokyo just the right time for a drinking party.&quot; It requires world knowledge to understand that there is a connection between how far away something is and the time that it will take to travel there. Although humans can make this inference intuitively, that understanding is not represented in the Gainen Base.</Paragraph> <Paragraph position="37"> Other incorrect judgments are the result of how the database handles determining the similarity scores, &quot;quirks&quot; that don't seem to match our intuitive judgments, as in (8): 8a.</Paragraph> <Paragraph position="38"> jikantekiniwa choudo taimingu wa iidarou shi timewise TOP exact timing TOP seems good and &quot;Timewise, the timing seems good, and the night game is not far away either.&quot; The Gainen Base yielded a relatively low score for the similarity between jiken, 'time,' and yoru, 'night,' despite our strong sense that these two words should be closely related.</Paragraph> <Paragraph position="39"> Overall, however, similarity scores seem to provide a fairly accurate measure of the relative coherence of this subset of the corpus. This result, coupled with the high 18 Recall that in 13 out of 15 cases, the Cf judged by humans to license a COHESIVE transition and the Cf picked out by the similarity measure matched. (7a) is one of the two clauses in which the human-chosen Cf did not match that chosen by the Gainen Base (the second is given in (8)). The previous clause is If it is a day game. The discourse entity tode, 'at a distance,' was the Cf chosen by human judgment to license the COHESIVE transition, since a human can make the connection that it is the day game that is at a distance (as evidenced in the translation). However, the use of the Gainen Base determined Tokyo to be more semantically similar to day game, with a score of 0.105 as compared to 0.005 for tode.</Paragraph> <Paragraph position="40"> Computational Linguistics Volume 30, Number 2 level of correlation discussed earlier between lexically cohesive entities designated by human judgment and those determined by semantic similarity, supports our proposal that lexical cohesion, as measured by semantic distance, can feasibly be included as a well-defined notion to capture crucial aspects of text coherence.</Paragraph> </Section> <Section position="5" start_page="140" end_page="141" type="sub_section"> <SectionTitle> 4.7 Exploring the Implications of the COHESIVE Transition and COMPLETE SHIFT </SectionTitle> <Paragraph position="0"> identity for Cbs has been recognized by Poesio et al. (2000) and others (Hahn, Markert, and Strube 1996; Murata, Isahara, and Nagao 1999). Poesio et al. supplemented the identity relation with three different possible semantic relations between Cfs in the utterances: set membership, subset, and &quot;generalized possession.&quot; Murata, Ishara, and Nagao induced a number of possible relations between bridging reference and anchor using a verb case frame dictionary and a corpus of Japanese AnoBexpressions (where A and B are nominal arguments and the AnoBconstruction encodes a wide variety of semantic relations between the two nominal arguments [Shimazu, Naito, and Nomura 1987]).</Paragraph> <Paragraph position="1"> As noted in Section 4.3.1, however, in all of these approaches, only a small subset of examples of bridging relations can be handled, because they all attempt to identify some particular relation existing between two elements. This is a necessary move for resolving bridging references and building text understanding systems, but neither of those is our aim here. We merely need to identify the level of semantic closeness or similarity between Cfs in U</Paragraph> <Paragraph position="3"> . Utilizing instead the more general semantic-distance measure proposed here, then, has several advantages over the explicit choice of particular relation labels. First, it avoids the need to make choices about which relations can or cannot, should or should not be included, as well as the difficulties with interlabeler reliability that Poesio et al. note, since there is no labeling in our approach. This is actually closely tied to another advantage: There is no need to limit the types of semantic relationships into which the Cfs can enter. Thus, there is no need to restrict our analysis to a subset of the phenomenon; our account will handle inferable centers having any kind of semantic relationship to the centers in the previous utterance.</Paragraph> <Paragraph position="4"> of COMPLETE SHIFTs. Is this number an accurate estimation of the (in)coherence in this corpus? Of course, as we saw in (5), when a message makes a shift in topic, we expect a COMPLETE SHIFT to occur. The writers of the messages in this corpus made 146 paragraphs (see Table 2); although we know that there is no guarantee that writers' paragraphing will coincide with shifts in cohesion, this number at least gives us a general estimation of a possible maximal number of COMPLETE SHIFTs (we would expect writers to err on the side of more paragraphs than topics rather than on the side of more topics than paragraphs). The number of COMPLETE SHIFTs is comfortably within that range.</Paragraph> <Paragraph position="5"> But there are two confounding factors that make this number higher than is actually appropriate for the nature of the corpus. The first is the inability of a centering approach to handle event deixis (Fais and Yamura-Takei 2003). Consider (9):</Paragraph> </Section> <Section position="6" start_page="141" end_page="143" type="sub_section"> <SectionTitle> Fais Transitions and Coherence </SectionTitle> <Paragraph position="0"> dou kahdo ni henkoushitai tonokotodesu same match-up to want to change thing is &quot;Because of U-san's urgent business, she would like to change to the same match-up on the 17th.&quot;</Paragraph> <Paragraph position="2"> &quot;How is it for everyone else?&quot; The zero argument translated as that in (9b) is an example of event deixis. When we examine the Cf list for (9a) to ascertain the Cb of (9b), we do not encounter that or its referent. In fact, we cannot encounter that. It is not possible for the discourse element represented by that to have appeared in the Cf list in (9a); clauses do not contain self-referential discourse elements. Thus, it is impossible to recognize, within the theory, that the discourse element that in (9b) is functioning as a strong cohesive element in the discourse. This is a problem to be resolved within centering theory regardless of whether COHESIVE transitions and COMPLETE SHIFTs are countenanced.</Paragraph> <Paragraph position="3"> However, sentential deixis does contribute to a slight skewing of the proportion of COMPLETE SHIFTs found in this corpus, for example, the COMPLETE SHIFT resulting from this problem in (9a)-(9b) and similar examples in the corpus.</Paragraph> <Paragraph position="4"> The second confounding factor is actually simply a byproduct of the nature of the corpus. Example (10), which is an entire message, illustrates: 10a. I-I-sanno teian de subete OK desu Computational Linguistics Volume 30, Number 2 The COMPLETE SHIFTs in this passage are perfectly appropriate; there is no cohesion among any of the discourse entities in this message. Why would anyone send a message that was completely, by this account, incoherent? In fact, the statements in (10) refer to the outcomes of various discussions held in the course of exchanging the messages in this corpus. This message occurs toward the end of the exchange of messages, as resolution of the questions of what sports event to go to, where to sit, and what restaurant to go to afterward is in sight. This writer is simply adding his opinions on each of these apparently unrelated topics. The relationship among them all holds only in the understanding of the coparticipants in the message exchange. Although we might be able to imagine the sorts of mechanisms required to model this level of understanding, we are a very long way from realizing them.</Paragraph> <Paragraph position="5"> transition into our centering account gives us a flexibility that is important for full-text understanding of the discourse. Consider example (11): &quot;Since I have seen it only on TV, I feel like watching a live game, but if it is on the 25th, M-san and H-san cannot join us for the game, can they?&quot; Lacrosse in (11a), (some) live game watching in (11b) and (11c), and (a) game (on the 25th) in (11d) are actually semantically distinct elements, and recognizing their distinctiveness is important for full-text understanding or summarizing. However, maintaining these distinctions in a standard account means characterizing the transitions between the utterances containing them as NULL SHIFTs. (It is not clear that we would even want to say that (a) game (on the 25th) was an inferable center from (some) live game watching.) Being able to designate these transitions as COHESIVE allows us both to maintain the 20 The message that this is taken from constitutes a sort of lesson on lacrosse from one of the authors to all the others. The [?] in (11a) refers to this global topic; however, neither lacrosse nor TV appears in the preceding utterance, and so (11a) has no Cb.</Paragraph> </Section> <Section position="7" start_page="143" end_page="143" type="sub_section"> <SectionTitle> Fais Transitions and Coherence </SectionTitle> <Paragraph position="0"> semantic distinctiveness of the discourse elements and to capture the coherence in this portion of the discourse.</Paragraph> <Paragraph position="1"> of some notion of lexical cohesion to delineate discourse structure is fairly well researched. In the lexical chain approach (Morris and Hirst 1991), a new discourse segment is hypothesized where the chain &quot;breaks,&quot; that is, where subsequent entities do not bear a semantic relationship to previous entities that would allow them to be added to the chain. Kozima (1993) provides an algorithm for determining where, on the graph of semantic cohesion values of words in a text, likely topic breaks occur and validates that determination against human judgment. We would say, then, that once a chain breaks or a significant dip in the semantic cohesion value graph occurs, the utterance following such a break is considered the first utterance of a new discourse segment.</Paragraph> <Paragraph position="2"> In terms of semantic distance as determined by the Gainen Base, we suggest that a sufficiently low similarity measure might characterize both COMPLETE SHIFTs and the beginning of a new discourse segment. Once again, &quot;sufficiently&quot; must be defined; in light of the preliminary results we saw previously, we conjecture that the definition of &quot;low&quot; will be relative to the measures for similarity in the message under scrutiny and not an absolute value (see Section 5.4).</Paragraph> <Paragraph position="3"> In addition, examination of the particular entities contributing to high levels of semantic similarity might also allow us to characterize the notion of &quot;global topic,&quot; albeit in a differentiated way. That is, if we determine not just one semantic distance measure for the sets of entities in two adjacent utterances, but the individual distances for each combination of those entities (as briefly described in note 13), we can determine those entities that are contributing the greatest amount of semantic similarity to the measure and identify a cluster of entities that can be taken to represent a global topic.</Paragraph> <Paragraph position="4"> This concept is worth examining more closely, since it bears on some of the very foundations of centering theory. The Cb of an utterance &quot;represents the discourse entity that the utterance U i most centrally concerns, similar to what is elsewhere called the 'topic'&quot; (Walker, Joshi, and Prince 1998, page 3). The presence of a Cb is taken to be both a necessary and a sufficient condition for topic coherence. However, in our account, utterances that have a coherent relationship to the immediate context of discourse may nonetheless have no Cb; that is, the presence of a Cb is, in our approach, only a sufficient condition. Our claim, then, is that this more accurately reflects the nature of how coherence is maintained in discourse: not only through the explicit repetition of a central entity, but also through the successive use of entities that are closely related semantically. In our account, Cbs are recognized and function just as in standard centering theory, but their absence, a common situation in at least some kinds of discourse, does not signal a breakdown in coherence. Coherence may be maintained as well by semantically similar entities, which can become Cbs in their own right, as match-up information and boxing do in (6). Using an individuated approach to determining semantic similarity, we can identify these particular entities.</Paragraph> </Section> </Section> class="xml-element"></Paper>