<?xml version="1.0" standalone="yes"?>
<Paper uid="P82-1034">
  <Title>DESIG~ DIMENSIONS FOR NON-NORMATIVE ONDERSTARDING SYSTEMS</Title>
  <Section position="1" start_page="0" end_page="0" type="metho">
    <SectionTitle>
DESIG~ DIMENSIONS FOR NON-NORMATIVE ONDERSTARDING SYSTEMS
</SectionTitle>
    <Paragraph position="0"/>
  </Section>
  <Section position="2" start_page="0" end_page="0" type="metho">
    <SectionTitle>
I. Introduction
</SectionTitle>
    <Paragraph position="0"> This position paper is not based upon direct experience with the design and implementation of a &amp;quot;non-normative&amp;quot; natural language system, but rather draws upon our work on cascade \[11\] architectures for understanding systems in which syntactic, semantic and discourse processes cooperate to determine the &amp;quot;best&amp;quot; interpretation of an utterance in a given discourse context. The RUS and PSI-KLONE systems \[I, 2, 3, 4, 5\], which embody some of these principles, provide a strong framework for the development of non-normatlve systems as illustrated by the work of Sondhelmer and Welschedel \[8, 9, 10\]and others.</Paragraph>
    <Paragraph position="1"> Here we pose a number of questions in order to clarify the theoretical and practical issues involved in building &amp;quot;non-normatlve&amp;quot; natural language systems. We give brief indications of the range of plausible answers, in order to characterize the space of decisions that must be made in deslgnlng such a system. The first questions cover what is intended by the ill-defined term &amp;quot;non-normatlve system&amp;quot;, beyond the important but vague desire for a &amp;quot;friendly and flexible&amp;quot; computer system. The remaining questions cover several of the architectural issues involved in building such a system, including the categories of knowledge to be represented in the system, the static modularization of these knowledge sources, and the dynamic information and control flow among these modules.</Paragraph>
    <Paragraph position="2"> The way the system is to deal with ill-formed input depends in a strong way on how much the system is expected to do with well-formed input.</Paragraph>
    <Paragraph position="3"> Ad hoc data base retrieval systems (a currently hot topic) pose different constraints than systems that are expected to enter into a substantlal dialogue with the user. When the behavior of the system is severely limited even given perfect input, the space of plausible inputs is also limited and the search for a reasonable interpretation for ill-formed input can be made substantially easier by asking the user a few well-chosen questions. In the dbms retrieval domain, even partially processed input can be used to suggest what information the user is interested in, and provide the basis for a useful clarification dialogue.</Paragraph>
    <Paragraph position="4"> What is the system expected to do with ill-formed input? The system may be expected to understand the input but not provide direct feedback on errors (e.g. by independently decldlng on the (most plausible) interpretation of the input, or by questioning the user about possible alternative interpretations). Alternatively, the system might provide feedback about the probable source of its difficulty, e.g. by pointing out the portion of the input which it could not handle (if it can be localized), or by characterizing the type of error that occurred and describing general ways of avoiding such errors in the future.</Paragraph>
    <Paragraph position="5"> 2. System performance goals What are the overall performance objectives of the system? Marcus has argued \[7\] that the &amp;quot;wellformedness&amp;quot; constraints on natural langua6e make it possible to parse utterances with minimal (or no) search. 2 The work we have done on the RU3 system has convinced us that this is true and that cascading semantic interpretatlon with syntactic analysis can further improve the efficiency of the overall system. The question naturally arises as to whether the performance characteristics of this model must be abandoned when the input does not satisfy the well-formedness constraints imposed by a competence model of language. We believe that it is possible to design natural language systems that can handle well-formed input efficiently and ill-formed input effectively.</Paragraph>
  </Section>
  <Section position="3" start_page="0" end_page="155" type="metho">
    <SectionTitle>
3. Architectural issues
</SectionTitle>
    <Paragraph position="0"> In order to design a fault-tolerant language processing system, it is important to have a model for the component processes of the system, how they interact in handling well-formed input, and how each process is affected by the different types of constraint vlolatlons occurring In 111formed input.</Paragraph>
    <Paragraph position="1"> What categories of knowledge are needed to understand well-formed input, and how are they used? Typically, a natural language understandlng system makes use of lexical and morphological knowledge (to categorize the syntactic and semantic properties of input items), syntactic knowledge, semantic knowledge, and knowledge of discourse phenomena (here we include issues of ellipsis, anaphora and focus, as well as plan  recognition (&amp;quot;why did he say this to me now?&amp;quot;) and rhetorical structure). Of course, saying that these categories of knowledge are represented does not imply anything about the static (representational) or dynamic (process interaction) modularizatlon of the resulting system.</Paragraph>
    <Paragraph position="2"> We will assume that the overall system consists of a set of component modules. One common decomposition has each category of knowledge embodied in a separate component of the NLU system, although it is possible to fuse the knowledge of several categories into a single prooeas. Given this assumption, we must then ask what control and information flow can be imposed on the interaction of the modules to achieve the overall performance goals imposed on the system. In analyzing how violations of constraints affect the operation of various components, it is useful to distinguish clearly between the information used wlthina component to compute its output, and the structure and content of the information which it oasses on to other components. It is also important to determine how critically the operation of the receiving component depends on the presence, absence or internal inconsistency of various features of the inter-component information flow.</Paragraph>
    <Paragraph position="3"> As an example, we will consider the interaction between a ayntactic component (parser) and a semantic interpretation component.</Paragraph>
    <Paragraph position="4"> Typically, the semantic interpretation process is componential, building up the interpretation of a phrase in a lawful way from the interpretations of its constituents. Thus a primary goal for the parser is to determine the (syntactically) acceptable groupings of words and constituents (a constituent structure tree, perhaps augmented by the equivalent of traces to tie together components). Unless such groupings can be made, there is nothing for the semantic interpreter and subsequent components to operate on. Some syntactic features are used only within the parser to determine the acceptability of possible constituent groupings, and are not passed to the semantic component (e.g. some verbs take clause complements, and require the verb in the complement to be subjunctive, infinitive, etc.). The normal output of the parser may also specify other properties of the input not immediately available from the lexical/morphological analysis of individual words, such as the syntactic number of noun phrases, and the case structure of clauses.</Paragraph>
    <Paragraph position="5"> Additionally, the parser may indicate the functional equivalent of &amp;quot;traces&amp;quot;, showing how certain constituents play multiple roles within a st,uc~ure, appearing as functional constituents of mi~c than one separated phrase. From the point of view of semantics, however, the grouping operation is of primary importance, since it is difficult to reconstruct the intended grouping without making use of both local and global syntactic constraints. The other results of the parsing process are less essential. Thus, for example, the case structure of clauses is often highly constrained by the semantic features of the verb and the constituent noun phrases, and it is possible to reconstruct it even with minimal syntactic guidance (e.g. &amp;quot;throw&amp;quot; &amp;quot;the bali&amp;quot; &amp;quot;the boy&amp;quot;).</Paragraph>
    <Paragraph position="6"> How can each component fill its role in the overall system when the constraints and assumptions that underlie its design are violated by ill-formed input? The distinction between the information used within a component from the information which that component is required to provide to other components is critical in designing processing strategies for each component that allow it to fulfill its primary output goals when its input violates one or more well-formedness constraints. Often more than one source of information or constraint may be available to determine the output of a component, and it is possible to produce well-formed output based on the partial or conflicting internal information provided by Ill-formed input. For example, in systems with feedback between components, it is possible for that feedback to make up for lack of information or violation of constraints in the input, as when semantic coherence between subject and verb is sufficient to override the violation of the syntactic number agreement constraint. When the integrity of the output of a component can be maintained in the face of ill-formed input, other components can be totally shielded from the effects of that input.</Paragraph>
    <Paragraph position="7"> A clear specification of the interface language between components makes it possible to have recovery procedures that radically restructure or totally replace one component without affecting the operation of other components. In general, the problem to be solved by a non-normative language understander can be viewed as one of finding a &amp;quot;sufficiently good explanation&amp;quot; for an utterance in the given context. ~ A number of approaches to this problem can be distinguished. One approach attempts ~o characterize the class of error producing mechanisms (such as word transposition, mistyping of letters, morphological errors, resumptive pronouns, etc.). Given such a characterization, recognition criteria for different classes of errors, and procedures to invert the error process, an &amp;quot;explanation&amp;quot; for an ill-formed utterance could be generated in the form of an intended well-formed utterance and a sequence of error transformations. The system would then try to understand the hypothesized well-formed utterance. While some &amp;quot;spelling corrector&amp;quot; algorithms use this approach, we know of no attempt to apply it to the full range of syntactic, semantic and pragmatic errors. We believe that some strategies of this sort might prove useful as components in a larger error-correcting system.</Paragraph>
    <Paragraph position="8"> A more thoroughly explored set of strategies for non-normative processing is based on the concept of &amp;quot;constraint relaxation&amp;quot;. If a component can find no characterization of the utterance because it violates one or more  constraints, then it is necessary to relax such constraints. A number of strategies have been proposed for relaxing well-formedness constraints on input to permit components to derive well-structured output for both well-formed and ill-formed input: 1. extend the notion of well-formed input to include (the Common cases of) ill-formed input (e.g. make the gr,m,~r handle ill-formed input explicitly); 2. allow certain specific constraints to be overridden when no legal operation succeeds; 3. provide a process that can diagnose failures and flexibly override constraints.</Paragraph>
    <Paragraph position="9">  Somehow the &amp;quot;goodness&amp;quot; of an explanation must be related to the number and type of constraints which must be relaxed to allow that explanation. How good an explanation must be before it is accepted is a matter of design choice. Must it simply be &amp;quot;good enough&amp;quot; (above some threshold), or must it be guaranteed to be &amp;quot;the best posslble&amp;quot; explanation? If it must be &amp;quot;the best possible&amp;quot;, then one can either generate all possible explanations and compare them, or use some strategy like the shortfall algorithm \[12\] that guarantees the first explanation produced will be optimal.</Paragraph>
    <Paragraph position="10"> While space prohibits discussion of the advantages and disadvantages of each of these strategies, we would llke to present a number of design dimensions along which they might be usefully compared. We believe that choices on these dimensions (made implicitly or explicitly) have a substantial effect on both the practical performance and theoretical interest of the resulting strategies. These dimensions are exemplified by the following questions: o Does the component have an explicit internal competence model that is clearly separated from its performance strategles? 4 o What information is used to determine which constraints to attempt to relax? Is the decision purely local (based on the constraint and the words in the immediate vicinity of the failure) or can the overall properties of the utterance and/or the discourse context enter into the decision? o When is relaxation tried? How are various alternatlves scheduled? Is it possible, for example, that a &amp;quot;parse&amp;quot; including the relaxation of a syntactic constraint may be produced before a parse that involves no such relaxation? o Does the technique permit incremental feedback between components, and is such feedback used in determining which constraints to relax? 5</Paragraph>
    <Section position="1" start_page="154" end_page="155" type="sub_section">
      <SectionTitle>
Non-syntactic ill-formedness
</SectionTitle>
      <Paragraph position="0"> While the overall framework mentioned above raises questions about errors that affect components other than syntax, the discussion centers primarily on syntactic ill-formedness. In this we follow the trend in the field. Perhaps because syntax is the most clearly understood component, we have a better idea as to how it can go wrong, while our models for semantic interpretation and discourse processes are much less complete. Alternatively, it might be supposed that the parsing process as generally performed is the most fragile of the components, susceptible to disruption by the slightest violation of syntactic constraints. It may be that more robust parslr~ strategies can be found. Without stating how the semantic component might relax its constraints, we might still point out the parallel between constraint violation in syntax and such semantic phenomena as metaphor, personification and metonymy. We believe that, as in the syntactic case, it will be useful to distinguish between the internal operation of the semantic interpreter and the interface between it and discourse level processes. It should also be possible to make use of feedback from the discourse component to overcome violations of semantic constraints. In the context of a waiter talking to a cook about a customer complaint, the sentence &amp;quot;The hamburger is getting awfully impatient.&amp;quot; should be understood.</Paragraph>
      <Paragraph position="1"> q. Conclusions We believe that it will be possible to design robust systems without giving up many valuable features of those systems which already work on well-formed input. In particular, we believe it will be possible to build such systems on the basis of competence models for various linguistic components, which degrade gracefully and without the use of ad hoc techniques such as pattern matching.</Paragraph>
      <Paragraph position="2"> One critical resource that is needed is a widely available, reasonably large corpus of &amp;quot;illformed input&amp;quot;, exhibiting the variety of problems which must be faced by practical systems. This corpus should be sub-divlded by modallty, since it is known that spoken and typewritten interactions have different characteristics. The collections that we know of are either limited in modality (e.g. the work on speech errors by Fromkin \[6\]) or are not widely available (e.g. unpublished material collected by Tony Kroch). It would also be valuable if this material were analyzed in terms of possible generative mechanisms, to provide needed evidence for error recovery strategies based on inversion of error generation processes.</Paragraph>
      <Paragraph position="3">  Finally, we believe that many error recovery problems can be solved by using constraints from one knowledge category to reduce the overall sensitivity of the system to errors in another category. To this end, work is clearly needed in the area of control structures and cooperative process architectures that allow both pipelinlng and feedback among components with vastly different internal knowledge bases.</Paragraph>
      <Paragraph position="4">  of course, on the overall goals of the system.</Paragraph>
      <Paragraph position="5"> ~In almost any case, we believe, the information available at the Interface between components should be expressed primarily in terms of some competence model.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>