<?xml version="1.0" standalone="yes"?>
<Paper uid="P86-1015">
  <Title>A MODEL OF REVISION IN NATURAL LANGUAGE GENERATION</Title>
  <Section position="3" start_page="90" end_page="91" type="metho">
    <SectionTitle>
2. DEFINING THE PROBLEM
</SectionTitle>
    <Paragraph position="0"> In order to make research into the problem of revision tractable, we need to first delimit the criteria by which to evaluate the text. They need to be broad enough to make a significant improvement in the readability of the text, narrow enough to be defined in terms of a representation a parser could build today, and have associated strategies for improvement that are definable in terms understood by the text planner and generator. In addition, we would like to delegate to the revision component those decisions which would be difficult for a generator to make when initially producing the text.</Paragraph>
    <Paragraph position="1"> As textual coherence often requires awareness of long range dependencies, we will begin by considering it an appropriate category of evaluation for a revision component.</Paragraph>
    <Paragraph position="2"> Coherence in text comes from a number of different sources. One is simply the reference made to earlier words and phrases in the text through anaphoric and cataphoric pronominal references; nominal, verbal and clausal substitution of phrases with elements such as 'one', 'do', and 'so'; ellipsis; and the selection of the same item twice or two items that are closely related. Coreferences create textual cohesion since the interpretation of one element in the text is dependent on another (Halliday and Hansan, 1976).</Paragraph>
    <Paragraph position="3"> Scinto (1983) describes a narrower type of cohesion which operates between successive predicational units of meaning (roughly clauses).</Paragraph>
    <Paragraph position="4"> These units can be described in terms of their &amp;quot;theme&amp;quot; (what is being talked about) and &amp;quot;rheme&amp;quot; (what is being said about it). Thematic progression is the organization of given and new information into theme-rheme patterns in successive sentences.</Paragraph>
    <Paragraph position="5"> Preliminary studies have shown (Glatt, 1982) that thematic progressions in which the theme of a sentence is coreferential with the theme or the theme of the immediately preceding sentence are easier to comprehend than those with other thematic progressions. This ease of comprehension can be attributed to the fact that the connection of the sentence with previous text comes early in the sentence. It would appear that the longer the reader must wait for the connection, the more difficult the integration with previous information will be.</Paragraph>
    <Paragraph position="6"> Another source of coherence is lexical connectives, such as sentential adjuncts ('first', 'for example', 'however'), adverbials ('subsequently', 'accordingly', 'actually'), and subordinate and coordinate conjunctions ('while', 'because', &amp;quot;but'). These connectives are used to express the abstract relation between two propositions explicitly, rather than leaving it to the reader to infer. Other ways of combining sentences can function to increase coherence as well. Chafe (1984) enumerates the devices used to combine &amp;quot;idea units&amp;quot; in written tex) including turning predications into modificatir  with attributive adjectives, preposed and postposed participles, and combining sentences using complement and relative clauses, appositives, and participle clauses. These structures function to increase connectivity by making the text more concise.</Paragraph>
    <Paragraph position="7"> Paragraph structure also contributes to the coherence of a text. &amp;quot;Paragraph&amp;quot; in this sense (Longacre, 1979) refers to a structural unit which does not necessarily correspond to the orthographic unit indicated by an indentation of the text.</Paragraph>
    <Paragraph position="8"> Paragraphs are characterized by closure (a beginning and end) and internal unity. They may be marked prosodically by intonation in speech or orthographically by indentation in writing, and structurally, such as by initial sentence adjuncts.</Paragraph>
    <Paragraph position="9"> Paragraphs are recursive structures, and thus may be composed of embedded paragraphs. In this respect they are similar to Mann's rhetorical discourse structures (Mann, 1984).</Paragraph>
  </Section>
  <Section position="4" start_page="91" end_page="92" type="metho">
    <SectionTitle>
3- A MODEL OF GENERATION AND REVISION
</SectionTitle>
    <Paragraph position="0"> In this section we will outline a model of generation with revision, focusing on improving textual coherence. First we estabLish a division of labor within the generation/revision process. Then we look at the phases of revision and consider the capabilities necessary for recognizing deficiencies in cohesion and how they may be repaired. In the fourth section, we apply this model to the revision of an example summary paragraph.</Paragraph>
    <Paragraph position="1"> The initial generation of a text involves making decisions of various kinds. Some are conceptually based, such as what information to include and what perspectives to take. Others are grammatically based, such as what grammatical form a concept may take in the particular syntactic context in which it is being realized, or how structures may be combined. Still others are essentially stylistic and have many degrees of freedom, such as choosing a variant of a clause or whether to pied pipe in a relative clause.</Paragraph>
    <Paragraph position="2"> The decisions that revision affects are at the stylistic level; only stylistic decisions are free of fixed constraints and may therefore be changed. Changes to conceptually dictated decisions would shift the meanin~ of the text. During initial generation, euristics for maintaining local cohesion are used, ~wing on the representations of simple local ~denctes. By &amp;quot;local&amp;quot;, we mean speciftcally that  we restrict the scope of information available to the generator to the sentence before, so that it can use thematic progression heuristics, letting revision take care of longer range coherence considerations.</Paragraph>
    <Paragraph position="3"> The revision process can be modeled in terms of three phases: I) recognition, which determines where there are potential problems in the text; 2) editing, which determines what strategies for revision are appropriate and chooses which, if any, to employ; 3) re-generation, which employs the chosen strategy by directing the decision making in the generation of the text at appropriate moments.</Paragraph>
    <Paragraph position="4"> This division reflects an essential difference in the types of decisions being made and the character of representations being used in each phase.</Paragraph>
    <Paragraph position="5"> The recognition phase is responsible for parsing the text and building a representation rich enough to be evaluated in terms of how well the text coheres. Since in this model the system is evaluating its own output, it need not rely only on the output text in making its judgements; the original message input to the generator is available as a basis for comparing what was intended with what was actually said. The goal is to notice the relationships among the things mentioned in the text and the degree to which the relationships appear explicitly. For example, the representation must capture whether a noun phrase is the first reference to an object or a subsequent reference, and if it is a subsequent reference, where and how it was previously mentioned. The recognition phase analyzes the text as it proceeds using a set of evaluation criteria. Some of these criteria look through the representation for specific flaws, such as ambiguous referents, while others simply flag places where optimizations may be possible, such as predicate nominal or other simple sentence structures which might be combined with other sentences. Other criteria compare the representation with the original plan in order to flag potential places for revision such as parallel sub-plans not realized in parallel text structure, or relations included in the plan which are expressed implicitly, rather than explicitly, in the text.</Paragraph>
    <Paragraph position="6"> Once a potential problem has been noted, the editing phase takes over. For each problem there is a set of one or more strategies for correcting it. For example, if there is no previous referent for the subject of a sentence, but there is a previous reference to the object, the sentence might be changed from active to passive; or if the subject has a relation to previous referent which is not explicitly mentioned in the text, more information may be added through modification to make that implicit connection explicit. The task of the editing phase is to determine which, if any, of these strategies to employ. (It may, for example decide not to take any action until further text has been analyzed.) However, what constitutes an improvement is not always clear. While using the passive may strengthen the coherency, active sentences are generally preferred over passives. And while adding more information may strengthen a referent, it may also make the noun phrase too heavy if there are already modifications. The criteria that choose between strategies must take into account the fact that the various dimensions along which the text may be evaluated are often in conflict. Simple evaluation functions will not suffice.</Paragraph>
    <Paragraph position="7"> The final step is actually making the change once the strategy has been chosen. This essentially involves &amp;quot;marking&amp;quot; the input to the generator, so that it will query the revision component at appropriate decision points. For example, if the goal is to put two sentences into parallel structure, the input plan which produces the structure to be changed would be marked. Then, when the generator reached that unit, it would query the revision component as to where the unit should be put in the text (e.g. a main clause or a subordinate one) and how it should be realized (e.g. active or passive).</Paragraph>
    <Paragraph position="8"> Note that as the revision process proceeds, it is continually dealing with a new text and plan, and must update its representations accordingly. New opportunities for changes will be created and previous ones blocked. We have left open the question of how the system decides when it is done.</Paragraph>
    <Paragraph position="9"> With a limited set of evaluation criteria, the system may simply run out of strategies for improvemenL The question will be more easily answered empirically when the system is implemented.</Paragraph>
    <Paragraph position="10"> An important architectural point of the design is that the system is not able to look ahead to consider later repercussions of a change; it is constrained to decide upon a course of action considering only the current state of the textual analysis and the original plan. While this constraint obviates the problems of the combinatorial explosion Of potential versions and indefinite lookahead, we must guard against the possibility of a choice causing unforeseen problems in later steps of the revision process. One way to avoid this problem is to keep a version of the text for each change made and allow the system to return to a previous draft if none of the strategies available could sufficiently improve the text.</Paragraph>
  </Section>
  <Section position="5" start_page="92" end_page="93" type="metho">
    <SectionTitle>
4. PARAGRAPH ANALYSIS
</SectionTitle>
    <Paragraph position="0"> In this section we use the model outlined above to describe how the revision component could improve a generated text. What follows is an example of the incremental revision of a summary paragraph. The discussion at each step gives an indication of the character of information needed and the types of decisions made in the recognition, editing, and regeneration phases.</Paragraph>
    <Paragraph position="1"> The example is from the UMass COUNSELOR Project, which is developing a natural language discourse system based on the HYPO legal reasoning system (Rissland, Valcarce, &amp; Ashley, 1984). The immediate context is a dialog between a lawyer and the COUNSELOR system. Based on information from the lawyer, the system has determined that the lawyer's case might be argued along the dimension &amp;quot;common employee transferred products or tools&amp;quot;. The system summarizes a similar case that has been argued along the same dimension as an example.</Paragraph>
    <Paragraph position="2"> The information to be included in the summary is chosen from the set of factual predicates that must be satisfied in order for the particular dimension to apply.</Paragraph>
    <Paragraph position="3"> In the initial generation of the summary, the overall organization is guided by a default paragraph organization for a case summary. The first sentence functions to introduce the case and place it as an example of the dimension in question. The body presents the facts of the case organized according to a partial ordering based on the chronology of the events. The final sentence summarizes the case by giving the action and decision. The choice of text structure is guided by simple heuristics which combine sentences when possible and choose a structure for a new sentence based on thematic progression, so that the subject of the new sentence is related to the theme or rheme of the previous sentence.</Paragraph>
    <Paragraph position="4">  (1) The case Telex vs. IBM was argued along the dimension &amp;quot;common employee transferred products or tools&amp;quot;. IBM developed the product Merlin, which is a disk storage system. Merlin competes with the T-6830. which was developed by Telex. The manager on the Merlin development project was Clemens. He left IBM in 1972 to work for Telex and took with him a copy of the Merlin code. IBM sued Telex for misappropriation of trade secret information and won the case.</Paragraph>
    <Paragraph position="5"> The recognition phase analyzes the text, looking for both flaws in the text and missed opportunities. The repetition of the word &amp;quot;develop&amp;quot; in the second and third sentences alerts the editing phase to consider whether a different word should be chosen to avoid repetition, or the repetition should be capitalized on to create parallel structure. By examining the input message, it determines that these clauses were realized from parallel plans, so it chooses to realize them in parallel structure.</Paragraph>
    <Paragraph position="6"> In the regeneration phase, the message is marked so that the revision component can be queried at the appropriate moments to control when and how the information unit for &amp;quot;Telex developed the T-6830&amp;quot; will be realized. After generation of the second sentence, the generator has the choice of attaching either &lt;develop Telex T-6830&gt; or &lt;compete Merlin T-6830&gt; as the next sentence. As one of these has been marked, the revision component is queried. Its goal is to make this sentence parallel to the previous one, so it indicates that the marked unit, &lt;develop ...&gt;, should be the next main clause and should be realized in the active voice. Once that has been accomplished, the default generation heuristics take over to attach &lt;competes with...&gt; as a relative clause: (2) The case Telexvs. IBM was argued along the dimension &amp;quot;common employee transferred products or tools&amp;quot;. IBM developed the product Merlin. which is a disk storage system. Telex developed the T-6830, which competes with Merlin. The menager on the Merlin development project was Clemens. He left IBM in 1972 to work for Telex end took with him a copy of the Merlin code. IBM sued Telex for misappropriation of trade secret information and won the case.</Paragraph>
    <Paragraph position="7"> Once the change is completed, the recognition phase takes over once again. It notices that sentence four no longer follows a preferred thematic progression as &amp;quot;Merlin&amp;quot; is no longer a theme or theme of the previous sentence. It considers the following possibilities: -- Create a theme-theme progression by moving sentence five before sentence four and beginning it with &amp;quot;Telex&amp;quot;, as in: &amp;quot;Telex was who Clemens worked for after he left IBM in 1972.&amp;quot; (Note there are no other possibilities for preferred thematic progressions without changing previous sentences.) -- Reject the previous change which created the parallel structure and go back to the original draft.</Paragraph>
    <Paragraph position="8"> -- Leave the sentence as it is. Although there is no preferred thematic progression, cohesion is created by the repetition of &amp;quot;Merlin&amp;quot; in the two sentences.</Paragraph>
    <Paragraph position="9"> -- Create an internal paragraph break by using &amp;quot;in 1972&amp;quot; as an initial adjunct. This signals to the reader that there is a change of focus and reduces the expectation of a strong connection with the previous sentences.</Paragraph>
    <Paragraph position="10"> The editor chooses the fourth strategy, since not only does it allow the previous change to be retained, but it imposes additional structure on the paragraph. Again during the regeneration phase the editor marks the information unit in the message which is to be realized differently in the new draft. Default generation heuristics choose to realize &amp;quot;Clemens&amp;quot; as a name, rather than a pronoun as it had been, and to attach &amp;quot;the manager...&amp;quot; as an appositive. (3) The case Telex vs. IBM was argued along the dimension &amp;quot;common employee transferred products or tools&amp;quot;. IBM developed the product Merlin, which is a disk storage system. Telex developed the T-5830, which competes with Merlin. In 1972. Clemens. the tanager on the Merlin development project, left IBM to work for Telex ud took with him * copy of the Merlin code. IBM sued Telex for misappropriation of trade secret information end won the case.</Paragraph>
  </Section>
  <Section position="6" start_page="93" end_page="94" type="metho">
    <SectionTitle>
5. OTHER REVISION SYSTEMS
</SectionTitle>
    <Paragraph position="0"> Few generation systems address the question of using successive refinement to improve their output. Some notable exceptions are KDS (Mann &amp; Moore, 1981), Yh (Gabriel, 1982), and Penman (Mann, 1983). KDS and Y=h use a top down approach where intermediate representations are evaluated and improved before any text is actually generated; Penman uses a cyclic approach similar to that described here.</Paragraph>
    <Paragraph position="1">  KDS uses a hill climbing module to improve text. Once a set of protosentences has been produced and grossly organized, the hill climber attempts to compose complex protosentences from simple ones by applying a set of aggregation rules, which correspond roughly to English clause combining rules. Next, the hill climber uses a set of preference rules to judge the relative quality of the resulting units and repeatedly improves the set of protosentences on the basis of those judgements. Finally, a simple linguistic component realizes the units as sentences.</Paragraph>
    <Paragraph position="2"> There are two main differences between this system and the one described in this paper. First, KDS uses a quantitative measure of evaluation in the form of preference rules which are stated independently of any linguistic context. The score assigned to a particular construction or combination of units does not consider which rules have been applied in nearby sentences. Consequently, intersentential relations cannot be used to evaluate the text for more global considerations. Secondly, KDS evaluates an intermediate structure, rather than the final text. Therefore, realization decisions, such as those made by KDS's Referring Phrase Generator, have not yet been made. This makes evaluating the strength of coherence difficult, since it is not possible to determine whether a connection will be made through modification.</Paragraph>
    <Paragraph position="3"> Yh also uses a top down improvement algorithm, however rather than having a single improvement module which applies one time, it evaluates and improves throughout the generation process. The program consists of a set of experts which do such things as construct phrases, construct sentences, and supply words and idioms. The &amp;quot;planner&amp;quot; tries to find a sequence of experts that will transform the initial situation (initially a specification to be generated) to a goal situation (ultimately text). First, experts which group the information into paragraph size sets are applied; then other experts divide those sets into sentence size chunks; next, sentence schemata experts determine sentence structure; and finally experts which choose lexical items and generate text apply. After each expert applies, critics evaluate the result and may call an expert to improve it. Like KDS, this type of approach makes editing of global coherence considerations difficult since structural decisions are made before lexical choices.</Paragraph>
    <Paragraph position="4"> The Penman System is the most similar to the one described in this paper. The principle data flow and division of labor into modules are the same: planning, sentence generation, improvement.</Paragraph>
    <Paragraph position="5"> However, an important difference is that Penman does not parse the text in order to revise it. Rather it uses quantitative measures, such as sentence length and level of clause embeddings to flag potential trouble spots. While this approach may improve text along some dimensions, it will not be capable of improving relations such as coherence, which depend on understanding the text. A similarity between Penman's revision module and the model described in this paper is that neither has been implemented.</Paragraph>
    <Paragraph position="6"> As the two systems mature, a more complete comparison may be made.</Paragraph>
  </Section>
class="xml-element"></Paper>