File Information
File: 05-lr/acl_arc_1_sum/cleansed_text/xml_by_section/metho/83/j83-3003_metho.xml
Size: 46,162 bytes
Last Modified: 2025-10-06 14:11:34
<?xml version="1.0" standalone="yes"?> <Paper uid="J83-3003"> <Title>Meta-rules as a Basis for Processing Ill-Formed Input 1</Title> <Section position="5" start_page="0" end_page="0" type="metho"> <SectionTitle> 3. Examples of Meta-rules </SectionTitle> <Paragraph position="0"> To further argue for recta-rules as a uniform framework for processing ill-formed input, we describe a wide variety of meta-rules in this section and the next.</Paragraph> <Paragraph position="1"> We adopt the following notation for meta-rules in this paper: Do not encode well-formedness constraints.</Paragraph> <Paragraph position="2"> Well-formedness rules convey meanings by constraining interpretations. Design systems for well-formed input and ill-formed input together. This gives no preference of well-formed interpretations over ill-formed ones. Search for well-formed interpretations prior to considering any ill-formed ones. This does not explicitly relate handling ill-formedness to processing well-formedness. Use a metric to rank ill-formedness interpretations, and select the one that comes closest to satisfying all constraints.</Paragraph> <Paragraph position="3"> This does not state what the deviation is so that one may draw inferences from the ill-formedness, nor does it capture actual patterns of error.</Paragraph> <Paragraph position="5"> The left-hand side (LHS) diagnoses what the problem might be; the right-hand side (RHS) states how to relax the failed constraint. The Ci are conditions on the computational state of the system; all must be true if the meta-rule is to apply. The Ai are actions stating how to rewrite the violated constraint and resume processing; all will be executed if the rule applies.</Paragraph> <Paragraph position="6"> Many of the actions we suggest here can be viewed as rewriting a rule of the normative system, for example, a grammar rule or case frame. However, some are more appropriately viewed as changing the computational state when blockage occurs; an example is correcting the spelling of a word. In Section 5 we will argue that it is best to implement all of the actions as modifications of a blocked alternative.</Paragraph> <Paragraph position="7"> Naturally, in rewriting a rule, pattern-matching and substitution are fundamental. We adopt the following definition of patterns. A pattern is a LISP sexpression. Atoms preceded by a question mark are variables. Expressions preceded by a dollar sign are evaluated using the LISP rules; their values are treated as patterns. 4 If a period appears before a pattern variable that is the last item in a list, that pattern variable matches the tail of a list. All other items in patterns are literals. The scope of a pattern variable is the whole meta-rule. The first time a variable is bound in a meta-rule, it retains the binding throughout the rule.</Paragraph> <Paragraph position="8"> Potentially, there may be many places where relaxation can occur. If a meta-rule applies to more than one configuration, it will be applied to each in turn, creating a list of possibilities for processing after recovery is complete. Consequently, the meta-rules will refer to only one failed configuration at a time.</Paragraph> <Paragraph position="9"> 3.1. Meta-rules related to syntax First, let us consider meta-rules dealing with the grammar. Many of our examples here are reformulations of our earlier work (Weischedel et al. 1978, Kwasny and Sondheimer 1979, Weischedel and Black 1980) within the uniform framework of meta-rules. All meta-rules pertaining to syntax should have a first condition which is (SYNTAX-FAILED?); this is true iff the parser is blocked. Since all rules in this section would contain that predicate, we will not include it in the examples. null Many syntactic formalisms have a similar framework for expressing rules: these include context-free grammars, augmented phrase structure grammars (Heidorn 1975), Programmar (Winograd 1972), the linguistic string parser (Sager 1981), Lifer (Hendrix et al. 1978), and augmented transition networks (ATNs) (Woods 1970). In fact, all of these can be viewed as formally equivalent to ATNs, and we will describe our techniques in that framework.</Paragraph> <Paragraph position="10"> Figure 1 gives several predicates that should be useful in the LHS of meta-rules. The LHS of the meta-rule is matched against the environment in which an ATN arc failed. The environment is called a configuration and includes the current ATN state, the arc, all ATN registers, and the remainder of the input string.</Paragraph> <Paragraph position="11"> 4 The expression $expr could be implemented as (*EVAL* expr). The pattern variable ?atom could be implemented as (*VAR* atom).</Paragraph> <Paragraph position="12"> Did the configuration block in state? Is the current word in category? Is the current word a member of list? Is the word after the current one in category? Is the word after the current one in list? Is the pattern a predicate expression in the test of the arc and did both the expression and the test evaluate to false? Does the failed arc match pattern? Is the hold list empty? Is x a word frequently confused with another? If so, the related word is returned. Figure I. Useful Conditions for Syntactic Meta-rules.</Paragraph> <Paragraph position="13"> American Journal of Computational Linguistics, Volume 9, Numbers 3-4, July-December 1983 165 Ralph M. Weischedel and Norman K. Sondheimer Meta-rules as a Basis for Processing Ill-Formed Input</Paragraph> <Paragraph position="15"> defines the hold list to be empty.</Paragraph> <Paragraph position="16"> adds the instantiation of the pattern to a list of violated constraints stored in the configuration. The position of the parser within the input string will automatically be recorded as well.</Paragraph> <Paragraph position="17"> changes the arc in the failed configuration by replacing all expressions matching patternl by pattern2.</Paragraph> <Paragraph position="18"> makes x the current word in the blocked configuration.</Paragraph> <Paragraph position="19"> An important action for the RHS is NEW-CONFIGURATION, which defines a new parser configuration, thus replacing the failed configuration that the meta-rule matched. It may take any number of arguments which set parts of the configuration. For example, SETR will define the new value of an ATN register. A list of useful arguments to NEW-CONFIGURATION is given in Figure 2. Failed constraints fill the role of the deviance notes of Kwasny and Sondheimer (1979). All parts of the failed configuration that are not explicitly changed in NEW-CONFIGURATION remain the same. Our implementation assumes that there is only one NEW-CONFIGURATION per meta-rule, though one could generalize this so that executing NEW-CONFIGURATION n times in a meta-rule gives n new configurations to replace the failed one. If a new configuration is generated, the parse can be resumed.</Paragraph> <Paragraph position="20"> Figure 3 gives a trivial ATN which will be used for the sample meta-rules. The start state is S/. A list beginning with an asterisk in the actions of an arc is a comment.</Paragraph> <Paragraph position="21"> 3.1.1. Simple grammatical tests Our earlier work showed how to relax tests that appear on ATN arcs. In one study (Eastman and McLean 1981), subject-verb disagreement occurred in 2.3% of the English queries. Meta-rule (i) relaxes that agreement test. The new configurations here are the result of replacing the agreement test in each failed arc by the predicate T. Since a new configuration is generated, parsing is resumed using it. Though the substitution was trivial in this case, SUBSTITUTE-IN-ARC is a general pattern-matching and substitution facility. As an example, consider &quot;A curious problem showing unusual conditions appear ...&quot; A top-down, left-to-right parse using a grammar such as the one in Figure 3 would block at the word &quot;appear&quot;. One of the blocked configurations would correspond to the agreement test failure in the arc leaving state S/NP; meta-rule (i) would apply, allowing the sentence to be parsed.</Paragraph> <Paragraph position="22"> Another frequently occurring problem is omitting required articles from count nouns. In the study by Eastman and McLean (1981) this occurred in 3.3% of all queries. In the grammar of Figure 3, blockage would occur at NP/N because of the test DET&NOUN-AGREE? on an example such as &quot;Print price of P27 over the last five years&quot;. Rule (ii) relaxes the test. When the parser starts on the new configuration, the modified test will be checked, verifying that no determiner is present. If none is, the message from FAILED-CONSTRAINT is available for error recovery.</Paragraph> <Paragraph position="23"> The meta-rule approach allows for more sophisticated actions. Suppose that a linguistic study of utterances with missing determiners showed that a default assumption of definite reference is a good heuristic. In this case, one could simply add the action (SETR DET 'the) to the actions in NEW-CONFIGURATION.</Paragraph> <Paragraph position="24"> One could argue that, in a data base environment, the grammar should simply treat omitted determiners as a normative construction. Even though determiners are frequently omitted in data base contexts, preferring well-formed interpretations can eliminate some ambiguities in complex noun phrases such as &quot;a machine running programs&quot;. The determiner constraint suggests that &quot;running programs&quot; modifies the head noun &quot;machine&quot; rather than &quot;machine&quot; and &quot;running&quot; both modifying &quot;programs&quot;.</Paragraph> <Paragraph position="25"> A number of word pairs are frequently confused, such as homonyms and &quot;good&quot; for &quot;well&quot;. Meta-rule (iii) allows for such errors, since REPLACE-* will modify the current word in the blocked configuration. MR-SETQ binds the value of its second argument to the pattern variable appearing as its first argument.</Paragraph> <Paragraph position="26"> Hence, &quot;You performed good&quot; would block at S/V, and the meta-rule would substitute &quot;well&quot; for &quot;good&quot;. Another kind of ill-formedness is resumptive pronouns and resumptive noun phrases. These occur in relative clauses where the entity referred to by the relative pronoun is improperly repeated in the relative clause as a pronoun or noun phrase. An example is &quot;John's friend Mary married the man that she planned to marry him&quot;, since there is no syntactic slot in the relative clause for the relative pronoun &quot;that&quot; to fill. A typical ATN strategy for interpreting relative clauses is to put a place holder or trace on a &quot;hold list&quot;; the ATN processor prevents POPping from a level if the hold list is non-empty. That test prevents accepting clauses where traces are not used. Meta-rule (iv) provides for resumptive pronouns and resumptive noun phrases.</Paragraph> <Paragraph position="27"> One can imagine more complicated tests, since there are specific conditions (Kroch 1981) under which resumptive pronouns and resumptive noun phrases are more likely.</Paragraph> <Paragraph position="28"> Of course, the parser may not be able to recover at all due to either absolute or relative ill-formedness.</Paragraph> <Paragraph position="29"> Weischedel and Black (1980) presented a technique for associating error messages with states where the parser blocked. The only way to block in S/POP is if the verb complement expected for the main verb is not present. Meta-rule (v) could handle this simple case.</Paragraph> <Paragraph position="30"> Notice that this is a different class of meta-rule, for it does not resume computation. Naturally, such rules should be tried only after no other meta-rules are available. One could define different classes of meta-rules by appropriate declarations; alternatively, this class can be recognized easily, since none of the actions resume processing.</Paragraph> <Paragraph position="31"> This is not the only alternative in the face of failure to parse even with relaxation; Jensen and Heidorn (1983) present heuristics for what to pass to the semantic interpreter in this case, given bottom-up parsing. null 3.2. Meta-rules related to semantics In addition to these syntactic examples, semantic problems can also be addressed within the formalism. If some semantic tests are included in the parser, say a certain arc test contains calls on the semantic component, specific semantic tests can be relaxed by the general mechanism we described for relaxing tests on ATN arcs.</Paragraph> <Paragraph position="32"> Instead, suppose that semantic constraints are encoded in a separate component. Semantic constraints may be expressed in several formalisms, such as semantic nets (Bobrow and Webber 1980a,b; Sondheimer et al. 1984), first-order logic, and production rules (for example, PROLOG, Warren et al. 1977). It is generally agreed that all are formally equivalent to first-order logic. For the purposes of this paper, we assume that the selection restrictions are encoded in first-order logic.</Paragraph> <Paragraph position="33"> One of the most common designs for a semantic interpreter is based on selection restrictions and case frames (Bruce 1975). At least five kinds of constraints may be violated: 1) what may fill a given case, 2) which cases are required for a complete constituent, null 3) which may have multiple fillers without conjunction, null 4) which are allowed for a given case frame, and 5) what order cases may appear in.</Paragraph> <Paragraph position="34"> such a semantic interpreter. Assume that any predicate on the semantic class of a constituent is encoded simply in LISP notation, for example, (HUMAN x) is true iff x is of class human. All meta-rules in this section can be assumed to include an initial test (SEMANTICS-FAILED?.) For convenience, we have used the same names for some of the actions as in the syntactic cases (for example, FAILED-CONSTRAINT, NEW-CONFIGURA-TION, etc.). When implemented in a particular system, different names may be used, since the concept of configuration, blockage, etc., is usually different for the types of processing (for example, lexical, syntactic, semantic, and pragmatic). Figure 5 lists several actions useful in semantic meta-rules.</Paragraph> <Paragraph position="35"> American Journal of Computational Linguistics, Volume 9, Numbers 3-4, July-December 1983 169 Ralph M. Weischedel and Norman K. Sondheimer Mete-rules as a Basis for Processing Ill-Formed Input</Paragraph> </Section> <Section position="6" start_page="0" end_page="0" type="metho"> <SectionTitle> CONCRETE ANIMATE INANIMATE /,,, HUMAN ANIMAL </SectionTitle> <Paragraph position="0"> Meta-rule (vi) is a very general rule. Assuming that semantic class tests are organized in a hierarchy, it states that the failed test is to be replaced by its parent in the hierarchy, yielding the next most general test.</Paragraph> <Paragraph position="1"> An example of the use of meta-rule (vi) is &quot;My car drinks gasoline&quot;. The restriction on the AGENT case could be the predicate ANIMATE. A fragment of a semantic hierarchy appears in Figure 6. In that, ANIMATE has a parent predicate of CONCRETE that would include cars. The failure of the initial sentence and the subsequent processing using the meta-rule would accept a sentence with the special deviance note identifying the semantic oddity.</Paragraph> <Paragraph position="2"> In a way similar to our arguments against approach 4 in Section 2.1, we feel that general meta-rules such as (vi) will prove less valuable than specific rules. A particular test that can be relaxed is the requirement for a human; for instance, the verbs of saying and those of propositional attitude, such as &quot;believe&quot; and &quot;think&quot;, normally have a restriction that their agent be human. Nevertheless, such a constraint is regularly violated through personification of pets, higher animals, machines, etc.</Paragraph> <Paragraph position="3"> Since personification is infrequent compared to the norm of descriptions designating humans, a case constraint of &quot;human&quot; can trim the search space. Since personification conveys particular inferences (Lakoff and Johnson 1980), a relaxation rule that records the detected personification can trigger appropriate inference processes. Figures of speech certainly are not absolutely ill-formed; we argue here that it is useful to treat them as relatively ill-formed.</Paragraph> <Paragraph position="4"> Meta-rule (vii) is one simple relaxation for personifying animals. More specific ones may prove preferable, if classes of personification are taxonomized. 3.2.3. Metonymy There are at least seven classes of metonymy (Lakoff and Johnson 1980), including a part for the whole, the producer for the product, the object for its user, the controller for the controlled entity, the institution for the people responsible, the place for the institution, and the place for the event. This analysis suggests two kinds of strategies. A particular class of descriptions may occur in exactly the same linguistic environments as their class of metonymous descriptions. For instance, institutions and people appear interchangeable as the logical subject of the verbs of saying and of propositional attitude. That can be encoded directly in the case frames of those verbs.</Paragraph> <Paragraph position="5"> However, many types of metonymy are conditioned on a highly specialized relationship. For instance, places can be used metonymously for events only if the speaker believes an event is identifiably associated with the location. For instance, compare the following examples: Pearl Harbor caused us to enter the war.</Paragraph> <Paragraph position="6"> *Fifth and Lombard caused us to reconsider graduated income taxes.</Paragraph> <Paragraph position="7"> Therefore, a meta-rule such as (viii) seems appropriate to prefer the normal, but accept metonymous descriptions of events by places. In meta-rule (viii), we have assumed that there is a variable FILLER of the semantic interpreter that holds the constituent to be assigned. Also, in the call to VIEWABLE?, CURRENT indicates that the pragmatic component should use its current context.</Paragraph> <Paragraph position="8"> Failure in selectional restrictions can indicate other semantic errors. These include ordering problems, for example, &quot;John killed with a gun Mary&quot;, and unexpected prepositions, for example, &quot;John killed Mary by a gun&quot;. The LHS of the appropriate meta-rules would begin with identification of selectional restriction failures but would also include other tests. The RHS would change the assumed case. A rule for the first example is (ix). Here the assumption is that the ordering problem will be first noted when &quot;Mary&quot; is tried as a time modifier. Using SUBSTITUTE-FOR-CASE postulates the constituent &quot;Mary&quot; to fill the object case and attempts to do so.</Paragraph> <Section position="1" start_page="0" end_page="0" type="sub_section"> <SectionTitle> 3.3. Generality of the approach </SectionTitle> <Paragraph position="0"> Though we have experience in implementing our framework for ATN parsers only, we believe the framework to be applicable over a broad range of parsers. It assumes only that a &quot;configuration&quot; or &quot;alternative&quot; representing a blocked, partial interpretation can be stored, modified, and restarted. No assumption regarding the direction of processing (for example, left to right), the nature of search (for example, top-down vs. bottom-up), nor the class of problem (for example, lexical, syntactic, or semantic) is made.</Paragraph> <Paragraph position="1"> For instance, the design of an implementation for semantic meta-rules as in Section 3.2 is complete. The underlying semantic component is based on searching case frames breadth-first with both top-down and bottom-up characteristics. Except for the one meta-rule regarding incorrect phrase ordering in Section 3.2.4, the semantic meta-rules themselves are independent of whether proposing a phrase for a given case in a frame is based on syntactic considerations or other criteria (for example, Schank et al. 1980). Naturally, the primitive conditions and actions of a given set of rules will depend on a particular formalism. In the next section, we relate our framework to a variety of parsers and problems.</Paragraph> </Section> </Section> <Section position="7" start_page="0" end_page="0" type="metho"> <SectionTitle> 4. Additional Supporting Evidence </SectionTitle> <Paragraph position="0"> Many natural language interfaces have some heuristics for processing one or more classes of ill-formed input.</Paragraph> <Paragraph position="1"> Since an exhaustive analysis would be impossible here, we will review only a handful of techniques that have inspired us to develop the meta-rule framework. We describe each technique by showing how it could be phrased as a meta-rule within our paradigm.</Paragraph> <Paragraph position="2"> The LADDER system (Hendrix et al. 1978) implements three major techniques for processing ill-formed input. All fit within the framework we suggest. One deals with recovery from lexical processing. In this system, the developer of a question-answering system prepares only a dictionary of well-formed words. If a sentence contains a word that is not in the dictionary, the parser will fail. The system localizes the area of failure to the ATN state associated with the partial interpretation that has proceeded rightmost in the input and that is shallowest (in terms of incompleted ATN PUSH arcs). Candidates for the correct spelling are limited to the words that would permit the parser to proceed and that are close to the spelling that appears. An equivalent meta-rule would check in the LHS that the parser failed. The RHS would compute a list of words expected next for each type of arc leaving that state, for example, the category members and literal words expected next. The next action would apply the Interlisp spelling correction algorithm to postulate a known word that was expected next. This word would replace the unrecognized one in the input and parsing would resume. A similar heuristic is running in our current implementation, with the addition that, if the unrecognized word appears to have an inflected ending, spelling correction is performed on the possible root.</Paragraph> <Paragraph position="3"> A second technique in LADDER deals with understanding contextual ellipsis, if no parse for the input is found. This heuristic interprets &quot;the fastest submarine&quot; as &quot;To what country does the fastest submarine belong&quot;, if it occurs after a query such as &quot;To what country does each merchant ship in the North Atlantic belong&quot;. In Weischedel and Sondheimer (1982), we extended that heuristic to allow for turn-taking in dialogues and to allow expansions as well as American Journal of Computational Linguistics, Volume 9, Numbers 3-4, July-December 1983 171 Ralph M. Weischedel and Norman K. Sondheimer Meta-rules as a Basis for Processing Ill-Formed Input substitutions, such as the elliptical form &quot;Last month&quot; following &quot;Did you go to Chicago?&quot; A third technique in LADDER is the printing of error messages, in the same sense that meta-rule (v) above prints a message when all attempts have failed.</Paragraph> <Paragraph position="4"> We could phrase this heuristic as a meta-rule whose LHS would check that the parser has blocked. This meta-rule would be ordered strictly after the ones for spelling correction and contextual ellipsis. A state would be postulated as the locale of the problem by the same heuristic as for spelling correction. The RHS would print for each arc that leaves that state the category, constituent, or word that was expected by that arc.</Paragraph> <Paragraph position="5"> Hayes and Mouradian (1980) emphasize recovery techniques for blocking during left-corner parsing (Aho and Ullman 1972). Their strategies are invoked only if the parser blocks. Two of them can be reformulated as meta-rules as follows. One meta-rule would check in its LHS that the parser was blocked and that a special parser variable (call it BLOCKED-PARSE) was empty. The RHS would save the blocked configuration in BLOCKED-PARSE, and start parsing as if the current word were the first word of the input. This would enable the system to ignore initial strings that could not be understood. A useful example of this is restarted inputs, such as &quot;Copy all print all headers of messages&quot;. A second meta-rule is related. The LHS would check whether the parser was blocked and BLOCKED-PARSE had a configuration in it. Furthermore, the LHS would check to see that another parser variable (call it DONE-ONCE) was NIL.</Paragraph> <Paragraph position="6"> If so, the RHS would set DONE-ONCE to T. The RHS would then swap the current configuration with BLOCKED-PARSE and would try resuming the parse from the current word with that configuration. This heuristic is designed to ignore incomprehensible material in the middle of an input. For instance, it would enable skipping the parenthetical material in &quot;List all messages, assuming there are any, from Brown&quot;.</Paragraph> <Paragraph position="7"> In the area of pragmatics, solutions that could fit within our paradigm have been suggested for two classes of problems. One problem is the failure of presuppositions of an input. In the environment of an intelligent tutor for computer-assisted language instruction, a technique suggested in Weischedel et al.</Paragraph> <Paragraph position="8"> (1978) could be formulated as a meta-rule as follows.</Paragraph> <Paragraph position="9"> The LHS would check whether processing was blocked due to a presupposition being false. Since that system would have a more complete knowledge of language than a beginning student of a foreign language, the system could treat the input as absolutely ill-formed.</Paragraph> <Paragraph position="10"> A sophisticated RHS could paraphrase the false presupposition for the student and indicate which word or syntactic construction was used inappropriately. Thus, the tutor could point out mistakes such as &quot;Das Fraeulein ist Student&quot;, indicating that the student should look up the meaning of &quot;Student&quot; (which applies only to males).</Paragraph> <Paragraph position="11"> Kaplan (1978) suggests an alternative heuristic for false extensional presuppositions in a data base environment. One can reformulate it as a meta-rule whose LHS would check that the query had requested a set as a response and that the set was empty. The RHS would compute queries corresponding to subsets that the original query presupposed would have a non-empty extension. The RHS would paraphrase the most general such query with an empty response set, reporting to the user that the system knew of no such entities. null</Paragraph> </Section> <Section position="8" start_page="0" end_page="0" type="metho"> <SectionTitle> 5. Implementation </SectionTitle> <Paragraph position="0"> We implemented a grammatical meta-rule processor first for an ATN interpreter and more recently for an ATN compiler (Burton and Brown 1977). Our experiments have used RUS (Bobrow 1978), a broad-coverage grammar of English with calls to a semantic component to block anomalous interpretations proposed by the grammar.</Paragraph> <Paragraph position="1"> Design and implementation of a meta-rule processor for violation of semantic constraints is currently underway in two different semantic interpreters. In one, case constraints are expressed as sets of logical formulas; in the other, KL-ONE is used to encode case frames (Sondheimer et al. 1984).</Paragraph> <Paragraph position="2"> Four design issues are considered in the following sections.</Paragraph> <Section position="1" start_page="0" end_page="0" type="sub_section"> <SectionTitle> 5.1. Applying meta-rules </SectionTitle> <Paragraph position="0"> The set of meta-rules dealing with the grammar or semantic system could be viewed formally as a function f from a component's rules S to a new component's rules S'.</Paragraph> <Paragraph position="1"> f(S) = S' S t is the transitive closure of applying every meta-rule pertaining to the system rules in every possible way. (Since it is the transitive closure, S is contained in S'). There are three alternatives. One is to compute S' and use it, rather than S, as the basis of processing, assuming that the transitive closure S' is a finite closure. The second is to apply meta-rules only as needed, thus making S ~ a virtual system. The third alternative is a combination of applying some meta-rules as needed and applying others in advance.</Paragraph> <Paragraph position="2"> The first alternative is superficially similar to approach 2 of Section 2.1, where ill-formedness processing is embedded in the normative system; however, S' will maintain the preference for normal interpretations over ill-formed ones. We have rejected this alternative because of the combinatorial growth of rules needed for S'. For instance, one can write meta-rules for handling relaxation of word categories and relaxation of predicates on ATN arcs. Since both can occur 172 American Journal of Computational Linguistics, Volume 9, Numbers 3-4, July-December 1983 Ralph M. Weischedel and Norman K. Sondheimer Meta-rules as a Basis for Processing Ill-Formed Input throughout the grammar, they should not be expanded ahead of time. A similar argument is used to justify treating conjunction processing as a separate process rather than building it directly into the grammar (Woods 1973). Since the classes of ill-formedness can occur in combination, the number of relaxed rules in S v can be very large. Furthermore, since utterances where many, many combinations of errors occur should be rare, computing the transitive closure seems uncalled for.</Paragraph> <Paragraph position="3"> The second alternative, generating a relaxed rule each time it is needed, is the one we implemented first in the context of an ATN interpreter. This alternative provides a kind of virtual system and avoids the increased memory necessary to hold S w.</Paragraph> <Paragraph position="4"> The third alternative, applying some rules ahead of time and using others only as needed, offers the greatest flexibility and a variety of alternatives. We have implemented a version in which the underlying parser is the output of an ATN compiler. When the meta-rule processor applies a meta-rule at a given arc, the relaxed version of the arc is compiled and saved. 5 If the meta-rule is to be tried by the meta-rule processor at that arc again, the form of the relaxed arc need not be re-computed; it can simply be executed.</Paragraph> <Paragraph position="5"> This third alternative also offers the potential of adapting the system to the idiosyncrasies of an individual's language and also the potential of extending its own model of language. Obviously, this is an area for future research.</Paragraph> <Paragraph position="6"> Alternatives two and three assume only that the processor applying well-formedness rules is able to store a &quot;configuration&quot; in a queue or agenda. No assumption about the type of processing (for example, bottom-up or top-down), nor the class of violated rule (for example, lexical, syntactic, semantic, or pragmatic) is necessary.</Paragraph> <Paragraph position="7"> 5.2. What to store When a configuration blocks because of the well-formedness rules, should the blocked configuration be stored or the results of applying each relevant metarule? Both of the implementations in the ATN environment save only the blocked configuration, namely, a blocked arc at the end of a path. The number of blocked configurations can be large. At present, there is insufficient evidence to determine whether a well-tuned set of meta-rules will yield a substantially larger (or smaller) number of relaxed configurations compared to the set of blocked configurations.</Paragraph> <Paragraph position="8"> Some types of problems, for example, subject-verb agreement, may be so common, and some types of 5 The current implementation is limited somewhat; it saves the relaxed arc only if the RHS of the meta-rule modifies only the arc itself. Our misspelling meta-rule, for example, does not modify the arc at all, but rather the input string.</Paragraph> <Paragraph position="9"> relaxation, for example, an unrecognized word, may be so diagnostically clear that the corresponding meta-rules should be applied immediately. In the case of subject-verb agreement, hand-compiling the meta-rule into the grammar may be appropriate (that is, writing an arc whose test is that subject-verb agreement failed and whose action places the new configuration on a queue that is tried only after all normal configurations have failed).</Paragraph> <Paragraph position="10"> 5.3. Localizing the problem When processing ill-formed inputs, some means of ranking alternatives is appropriate, since the system must determine what is intended in the face of violated constraints and possible error. Also, the number of relaxed configurations may be large, even with a set of well-tuned meta-rules designed to open the search space minimally.6 The ideal solution is that the ranking of alternatives should be based on syntactic, semantic, and pragmatic evidence, in addition to the diagnosis and recovery strategy.</Paragraph> <Paragraph position="11"> The current implementation uses only some of those bases and employs a rather simple ranking.</Paragraph> <Paragraph position="12"> Since both grammatical constraints and selection restrictions are employed while parsing with RUS, both syntactic and semantic evidence is used. Blocked configurations are ordered on the amount of input processed; there is also a partial order on the meta-rules. One of our students, Amir Razi, is designing an experiment to collect data on the performance of the system. The current system can be run in one of two modes: saving all blocked configurations or using only ones that proceeded rightmost in the input. One aspect of the experiment is to determine the frequency with which the interpretations covering the most input in a left-to-right parse block at the true source of the problem. Some preliminary evidence (Weischedel and Black 1980) indicates that this heuristic frequently does indicate where the problem is, if the normative system is nearly deterministic, for example, because the grammar is a fairly constrained subset of English or because semantic criteria filter out parses that have no meaning in the application domain.</Paragraph> <Paragraph position="13"> Our long-term goal is accurate determination of both the problem in an ill-formed utterance and what was intended. The current implementation represents the first step toward that by employing both syntactic and semantic evidence. We are investigating the use of pragmatic evidence for that purpose as well. In addition, we wish to explore techniques for examining both the left and right contexts of a blocked interpretation, for instance, by employing bottom-up processing. null 6 However, it is not clear whether the combinatorics alone for typical inputs will be a problem, given the rapid increase in processor power/cost and the prospect of multi-processing. American Journal of Computational Linguistics, Volume 9, Numbers 3-4, July-December 1983 173 Ralph M. Weischedel and Norman K. Sondheimer Mete-rules as a Basis for Processing Ill-Formed Input 5.4. A mete-rule index Hand-compilation of meta-rules as mentioned in Section 5.1 is just one way to pinpoint the configurations to which a meta-rule applies; another is providing an index from blocked configurations to the meta-rules that could apply. We have implemented a preprocessor that builds an index from an ATN arc to the meta-rules that can apply to it. When loading the ATN grammar, our preprocessor localizes the syntactic meta-rules having IN-STATE?, FAILED-TEST?, and FAILED-ARC? in their LHS to the few arcs to which they could possibly apply. Clearly, if IN-STATE? is in the LHS, that meta-rule can apply to only the handful of arcs leaving one state. Since FAILED-ARC? and FAILED-TEST? require the arc to match a given pattern, meta-rules using these tests can be identified with the arcs satisfying those patterns. 7 Such preprocessing provides an index into the possible rules that apply to a blocked configuration, since the state and the arc will be part of the configuration. Furthermore, the pattern-matching operations in the LHS need not be repeated at run-time, since the preprocessor stores for each arc an altered form of the meta-rule (without the calls to the pattern matcher) and the bindings that pattern matching created.</Paragraph> <Paragraph position="14"> Some meta-rules will not have any tests that localize their applicability; an example is the one for confusion words, which can appear almost anywhere. These are stored separately, and must be checked for any arc to which relaxation is to be tried.</Paragraph> </Section> </Section> <Section position="9" start_page="0" end_page="0" type="metho"> <SectionTitle> 6. Limitations </SectionTitle> <Paragraph position="0"> There are a number of points of caution. It should be clear that relaxation does not necessarily guarantee understanding. After all, relaxing any arc to (TST X T ...) will accept any word syntactically; yet that is no guarantee that the word will be understood. Relaxing constraints introduces additional potential for confusion. null What one classifies as &quot;absolutely ill-formed&quot; is clearly open to dispute, as Ross (1979) points out.</Paragraph> <Paragraph position="1"> Therefore, the system may classify something as illformed, ranking it behind other interpretations, even though the user views it as well-formed. We suspect that categorizing almost any particular constraint as normative could be the basis for argument. The criteria for deciding whether a constraint should be included in the normative system should include at least the following: a) whether a native speaker would edit inputs that violate it, b) whether violating the constraint can yield useful inferences, 7 Of course, this preprocessing assumes that no patterns in the LHS contain a form $expr.</Paragraph> <Paragraph position="2"> c) whether examples exist in which the constraint carries meaning, d) whether the constraint, if classified as normative, trims the search space, and e) whether a processing strategy for the constraint can be stated more easily as a modification of normative processing, as in the case of conjunction (Woods 1973) or the case of contextual ellipsis in the data base environment (Weischedel and Sondheimer, 1982).</Paragraph> <Paragraph position="3"> Thus far we have considered only constraints that are associated with a single point in the processing, such as relaxing a single case frame or relaxing a single ATN arc. Obviously, this need not be the case if, for instance, word or phrase order is permuted. At present, we have no general way of dealing with such problems.</Paragraph> </Section> <Section position="10" start_page="0" end_page="0" type="metho"> <SectionTitle> 7. Future Work </SectionTitle> <Paragraph position="0"> The problems of processing ill-formed input require several substantial research efforts. One is collecting additional corpora to determine patterns of errors and their frequency of occurrence. This is particularly important for two reasons. First, the more detail uncovered on patterns of error, the tighter the meta-rules for relaxing constraints. In our experience, the effort to make relaxation procedures as constrained and accurate as warranted by the patterns of occurrence is highly worthwhile, not only in trimming the search space, but also in eliminating senseless interpretations.</Paragraph> <Paragraph position="1"> Second, the patterns of ill-formedness will depend on the user community and the modality of input. For instance, non-native speakers of a language make different errors than native speakers. Typed input has a predominance of typographical/spelling errors; spoken input may have more restarted utterances.</Paragraph> <Paragraph position="2"> As a correlate to the need for more corpora of ill-formed natural language, there is an obvious need to define highly specific heuristics (as meta-rules) to diagnose and recover from each type of ill-formedness.</Paragraph> <Paragraph position="3"> Some of the heuristics should involve clarification dialogue, another area for research.</Paragraph> <Paragraph position="4"> There are many possible responses given a diagnosed problem. Consider a simple problem: violation of selection restrictions. In German, the verb &quot;fressen&quot; presupposes that the one eating is an animal. To an input such as &quot;Dieser Mann frisst oft&quot;,8 several recovery strategies could apply: a) The selection restriction could be ignored.</Paragraph> <Paragraph position="5"> b) The selection restriction could be generalized for future use.</Paragraph> <Paragraph position="6"> c) The system could conclude that an error has occurred, as in the aforementioned language learning environment.</Paragraph> <Paragraph position="7"> 8 &quot;This man eats often.&quot; 174 American Journal of Computational Linguistics, Volume 9, Numbers 3-4, July-December 1983 Ralph M. Weischedel and Norman K. Sondheimer IVleta-rules as a Basis for Processing Ill-Formed Input d) The system could engage in clarification dialog to determine whether the user intended to use that word.</Paragraph> <Paragraph position="8"> e) The system could assume the user believes that the man referred to eats like an animal.</Paragraph> <Paragraph position="9"> The conditions for selecting a strategy need to be studied. An explicit model of the user is needed for deciding the intent of the user and for appropriate recovery from ill-formedness.</Paragraph> <Paragraph position="10"> Learning the idiosyncrasies of particular users and automatic extension of the system (based on detecting relatively ill-formed input) is very challenging. Some initial steps in this direction have been taken in Carbonell (1979), but there is much to be done. A significant aspect of the learning problem in this environment is the substantial uncertainty about whether the system has the intended interpretation, and the effect on both the functional and time performance of the system as the abnormal is viewed as more normal (and the search space correspondingly grows).</Paragraph> <Paragraph position="11"> For syntactic iU-formedness, pure bottom-up parsing is intuitively very appealing, since one has descriptions of what is present both to the left and the right of the problem(s). The EPISTLE project (Jensen and Heidorn 1983) is employing bottom-up parsing. The advantage of employing top-down strategies, including left-corner parsing strategies, is the strong expectations available when a configuration blocks. Consequently, many relaxation strategies and systems in the literature (for example, Hendrix, et al. 1978; Kwasny and Sondheimer 1981; Weischedel and Sondheimer 1982) have been proposed and implemented in that framework. Use of bottom-up strategies offers interesting new classes of relaxation, such as rearranging constituents for ordering problems. It is not obvious how the combinatorics of bottom-up strategies will compare to those of top-down strategies. However, developing relaxation techniques for bottom-up processing and extensive empirical studies comparing them to top-down are certainly needed.</Paragraph> <Paragraph position="12"> One of the most critical problems is control. The need to relax the very rules that constrain the search for an interpretation is like opening Pandora's box.</Paragraph> <Paragraph position="13"> This affects not only the time required to understand an ill-formed input, but also ambiguity through the additional alternatives the system is prepared to accept. There are several aspects to controlling this search. First, the well-formedness constraints should reflect strictly what is normative. Second, the relaxation rules should be made as tight as warranted by patterns of ill-formedness in language use. Third, a partial order on the relaxations should be established.</Paragraph> <Paragraph position="14"> Fourth, not only syntactic constraints and selection restrictions should be used (as in our system) but also pragmatic information to suggest the most promising alternatives. We have begun research on how to use pragmatic knowledge in an information-seeking environment for this purpose; see Carberry (1983, 1984) and Ramshaw and Weischedel (1984). In the environment of messages reporting events, Granger (1983) reports on using expectations based on stereotypical events for this purpose. Extensive empirical studies regarding effective control of the search space are needed.</Paragraph> <Paragraph position="15"> The acid test for a framework, relaxation heuristics, and control strategies is not relaxing simple tests like subject-verb agreement or diagnosing obvious problems like a word not in the dictionary. Rather the acid test is a wide spectrum of problems, including examples like misspellings/typographical errors that result in a known word, because in this type of example, all of the local evidence can indicate that the incorrect word is perfectly correct. Trawick (1983) has initiated work on such misspelling problems.</Paragraph> </Section> class="xml-element"></Paper>