<?xml version="1.0" standalone="yes"?>
<Paper uid="C80-1008">
  <Title>A RULE-BASED APPROACH TO ILL-FORMED INPUT</Title>
  <Section position="3" start_page="46" end_page="48" type="metho">
    <SectionTitle>
3. A Rule-Based Approach to Ill-formed Input
</SectionTitle>
    <Paragraph position="0"> Out of the options available for preparing for and responding to ill-formed input, we propose one,in particular. This section begins with a short statement of our proposal and continues by clarifying and motivating it. Evidence for it from other work is then presented.</Paragraph>
    <Section position="1" start_page="46" end_page="46" type="sub_section">
      <SectionTitle>
3.1 Statement
</SectionTitle>
      <Paragraph position="0"> In essence, we believe that ill-formedness should be treated as rule-based. We see two kinds of rules: first, r~s used in normal processing and second, meta-rules which are only employed to interpret ill-formed input. With respect to the first, we feel that their violation should be used to detect ill-formed input. With respect to the second, we feel that they should be meta-rules applying to the rules of the first sort in order to relate the structure of ill-formed input to that of well-formed structures. This would be done by showing how the well-formedness rules could be modified to accept the ill-formed input with as complete a structure as possible. They will indicate a general type of error user's make.</Paragraph>
      <Paragraph position="1"> In terms of the three phases discussed in the last section, acceptance of our conjecture would lead to separate development of components for handling well-formed and ilL-formed inputs.</Paragraph>
      <Paragraph position="2"> Considering syntactic processing as an example, a normative gransnar would be written to interpret grammatically well-formed sentences.</Paragraph>
      <Paragraph position="3"> Separately, meta-rules would be developed for grammatically ill-formed sentences.</Paragraph>
      <Paragraph position="4"> Error identification would include analysis of failures in normal processing rules using the rules defining ill-formedness. For example, an error identification component would find the cause of a blockage in parsing by considering the failed grammar rules and the meta-rules that show how these normative rules could fail. In light of these, the normative rules could be modified automatically via the meta-rules in order to see if the input could be accepted. null Finally, whenever error recovery was feasible, it wou\]d use the ill-formedness rules to guide the modification of the rules of normal processing in order to continue processing the ill-formed input. For example, a failed semantic restriction test can be relaxed by a meta-rule and processing continued. Note that this often introduces uncertainty in that the constraint often carries semantic information, hence complete understanding is not guaranteed by our proposal.</Paragraph>
    </Section>
    <Section position="2" start_page="46" end_page="48" type="sub_section">
      <SectionTitle>
3.2 Example
</SectionTitle>
      <Paragraph position="0"> Consider subject-verb number agreement as in Weischedel (1977). Presumably any natural language interface for well-structured input would have tests to check for this, since it reflects semantic information, e.g., verb number differentiates between the meanings of &amp;quot;Flying planes is dangerous&amp;quot; and &amp;quot;Flying planes are dangerous.&amp;quot; However, number agreement errors are known to occur. We would capture this by adding a meta-rule allowing the agreement test to be ignored. This would, of course, be done at the cost of not identifying the intended sense. According to our proposal, an input, such as &amp;quot;The boy run fast&amp;quot;, would be treated as a potentially well-formed until the granmaar failed to interpret it. When the example fails to parse, we would attempt identification of the input based on the failure of the agreement test and the meta-rule. Then recovery would be attempted by removing the test and proceeding without knowing whether singular or plural was intended.</Paragraph>
      <Paragraph position="1"> Of course a system could at this point request user supplied clarification, or it could decide to abort the processing. However, our goal is to provide the ability to automatically  interpret as much as possible.</Paragraph>
    </Section>
    <Section position="3" start_page="48" end_page="48" type="sub_section">
      <SectionTitle>
3.3 AssL~nptions
</SectionTitle>
      <Paragraph position="0"> Underlying our belief in the viability of this approach to ill-formedness are some assumptions that limit the problem. Most important is the assumption of a cooperative user. Observation of cooperative users has shown that they tend to keep their requests linguistically simple and tailored to what they feel are the system's limits (Woods, 1973; Malhotra, 1975; Damerau, 1979). At the same time, users have been shown to be able to communicate effectively through limited machine interfaces (Kelly and Chapanis, 1977). This allows us to ignore many of the more difficult ill-formedness phenomena.</Paragraph>
      <Paragraph position="1"> An uncooperative user could &amp;quot;break&amp;quot; any system. For example, a user is reported to have asked a well-known system &amp;quot;what the h-ll is going on here?&amp;quot;. No system should be expected to handle such 'f input.</Paragraph>
      <Paragraph position="2"> Overshoot is a related phenomenon.</Paragraph>
      <Paragraph position="3"> Overshoot often arises with users unfamiliar with the capabilities of the computer system underlying the natural language interface (Woods, 1973; Shneiderman, 1978; Tennant, 1979).</Paragraph>
      <Paragraph position="4"> In order to allow for any overshoot we must be able to depend on our understanding of the user's knowledge. We therefore assume that the user has at least basic familiarity with the purpose and power of the underlying system.</Paragraph>
      <Paragraph position="5"> Finally, we assume that the natural language interface for normal sentences is well-structured in the sense of handling like sentences similarly and unlike sentences dissimilarly, and in the sense of having a decomposition of processing into explainable and defensible phases. In progra~ninglanguages, it is the case that grammars and parsers can be written to identify and recover from errors (Aho and Johnson, 1974). This ought to be the case with natural language interfaces. We are willing to defend our conjecture independent of any one structuring as long as the interface for well-formed input we are augmenting is built on consistent, explainable lines.</Paragraph>
    </Section>
  </Section>
  <Section position="4" start_page="48" end_page="49" type="metho">
    <SectionTitle>
4. Supporting evidence
</SectionTitle>
    <Paragraph position="0"> We will now consider evidence supporting our proposal.</Paragraph>
    <Section position="1" start_page="48" end_page="48" type="sub_section">
      <SectionTitle>
4.1 Pragmatic Motivation
</SectionTitle>
      <Paragraph position="0"> There are a number of reasons to prefer this solution, independent of the empirical evidence that we will present shortly. Basically, this approach will ease systems development and processing. This is true first because of the ability to design the normative processing system independent of the error identification and recovery methods. Second, not invoking ill-formedness processing until normal processing fails avoids unnecessary runtime costs for well-formed sentences, which are the normal type of input. Third, describing ill-formedness through meta-rules that relate to normative rules will avoid duplication of aspects of normative processing and allow general statements covering classes of ill-formedness.</Paragraph>
    </Section>
    <Section position="2" start_page="48" end_page="49" type="sub_section">
      <SectionTitle>
4.2 External Supporting Evidence
</SectionTitle>
      <Paragraph position="0"> There is support for our proposal from many other areas where ill-formedness in natural or artificial languages is considered. Most relevant are the efforts of linguists. When they have considered ill-formedness it has been con~non for them to propose the type of meta-rules we propose. For example, Chomsky (1964) relates failures to abide by different aspects of his gran~nar model to different classes of ill-formedness through relaxation of well-formedness constraints. Linguists also try to spot patterns in utterances containing errors in order to motivate rules for normal processing (Fromkin, 1973).</Paragraph>
      <Paragraph position="1"> A pattern of rule-based treatment of ill-formedness can be seen elsewhere. In information retrieval, index terms are processed as if they were correctly presented, until failure starts recovery methods based on rules which change the conditions for acceptance (Damerau, 1964). In progranmling languages, similar processing is seen with typographic errors and with syntactic problems such as incorrect numbers of parentheses (Teitelman, 1969; Morgan, 1970; Aho and Johnson, 1974). Trapping based on normative constraints and error recovery (at least in notifying the user) is seen in the maintenance of data base integrity (Wilson and Salazar, 1979).</Paragraph>
      <Paragraph position="2"> Finally, speech understanding systems, whose ill-formedness problems are related to noisy signals, often work from an initial assumption that a clear interpretation can be found for the input. When this fails, they take what they have found and attempt to recover by applying normative rules in a less rigorous way in order to identify the ill-formed segments (Bates, 1976; Miller, 1974).</Paragraph>
      <Paragraph position="3"> 4..3 Support from Natural Language Interface  EfTor~--To our knowledge, our general approach to ill-formedness has not been propounded elsewhere. However, work fitting within the paradigm has been applied to a number of isolated ill-formedness problems. In addition, one important technique which has been employed for ill-formedness appears to be modifiable so as to fit within our approach. The success of these efforts stands as support for our approach. In this section, they will be briefly surveyed.</Paragraph>
      <Paragraph position="4">  A lexicon may be thought of as a computa- null tional model of dictionary information. According to our approach, p~ocessing of lexical ill-formedness would be developed separately from the preparation of the processing of normal lexical entries (i.e. dictionary entries). Once the rules for processing well-formed inputs fai\] to recognize a lexical entry, error identification would begin based on the failed rules and rules which showed how lexical entries could be ill-formed. At the end of this identification phase, a guess or guesses as to the identity of a lexical entry would be available for the system to attempt recovery. This paradigm for processing can be seen in a number of systems in attempts to treat both absolute and relative lexical ill-formedness.</Paragraph>
      <Paragraph position="5"> The LIFER system is prepared to deal with misspelled and mistyped words through a method fitting within our model (Hendrix et al., 1978). The developer of a question-answering system using LIFER prepares only a dictionary of well-formed words. If a sentence contains a word that is not in the dictionary, the LIEER parser will fail and start error identification. LIFER first chooses as the putative failed rule the one associated with the partial interpretation that has proceeded furthest. From that rule, LIFER identifies the part of speech the word should belong to and applies a mistyping and misspelling algorithm based on such meta-rules as &amp;quot;expect letters to be duplicated&amp;quot; or &amp;quot;expect letters to be reversed&amp;quot; to modify the normal dictionary look up rulesand to match the ill-formed input to all well-formed members of the desired part of speech. If one is found, normal processing resumes.</Paragraph>
      <Paragraph position="6"> Examples related to our approach can also be seen in methods that deal with relative illformedness. For example, Granger's (1977) FOUL-UP program proceeds through input until it finds an unknown word. Based on its expectations for the input derived from parsing and its model of semantic content, it attempts recovery by assigning a partial interpretation to the input. null Somewhat similar processing can be seen in dealing with typographic errors (Biermann and Ballard, 1978), learning new names (Codd et al., 1978), and learning new words (Carbonell, 1979 , and Miller, 1975).</Paragraph>
      <Paragraph position="7">  With syntactic processing, our paradigm calls for separate development of a gra~ar for well-formedness, identification of errors based on the failure to parse, and error recovery based on manipulation of the grammar. This is most clearly seen in our own work. weischedel (1977) was the first to suggest several different techniques for dealing with syntactically ill-formed input. One technique allows gra~ar writers to insert rules to enable selective relaxation of restrictions in the gran~ar so that certain ungrammatical sentences may be assigned as much structure as possible. For example, his method would allow the number-agreement test to be relaxed as was discussed. Weischedel's method was tested in a natural language understanding system for intelligent tutoring of students learning a foreign language (Weischedel et al., 1978). A second technique suggested by Weischedel (1977) is the assignment of meanings to the states of an ATN grammar. These assignments were used to guide error identification for the end-user when interpretation of a sentence blocked at a state. The assignments could be quite general including operational procedures and could attempt complex deductions of the source of the error. Weischedel and Black (1980) report the results of testing the method on a parser for English.</Paragraph>
      <Paragraph position="8"> Kwasny and Sondheimer (1979) extend Weischedel's first method to allow for successively less stringent constraints. In addition, they propose a relaxation method using hierarchical structuring of syntactic categories, based on a suggestion in Chomsky (1964). If the normal rules fail to accept a sentence and the failed rule is looking for a part of speech which is a member of a hierarchy, then relaxation proceeds by substituting the next more general class in the hierarchy for the unsatisfied part of speech.</Paragraph>
      <Paragraph position="9"> Perhaps the most powerful technique of treating syntactic ill-formedness, as Hayes and Reddy (1979) and Hayes and Mouradian (1980) point out, is including patterns for ill-formed input. Kwasny and Sondheimer (1979) generalize this technique by allowing evenmore dramatic relaxation of the grammar through patterns that allow the input to be matched against the grammar in a relaxed way, either by skipping words in the input, or by skipping the application of rules. This is most useful for assigning structure to sentence fragments. Importantly, it also applies to many types of conjunction including the problematic case of gapping, cases. This technique dif\[ers from the paradigm suggested here because of its method of error identification and recovery. When an input is not recognized by the gran~ar, processing switches to an entfrely separate set of arcs in an ATN gra~runar, essentially another grammar, which are used to assign structure to the ill-formed input. However, experience with the method suggests that the arcs used in this separate grammar could in general be found in the normative grammar. If this is always the case, then the separate gran~aar could be eliminated. Also, error identification could proceed by considering the failure of the normative rules; error recovery could proceed by relaxing the conditions on the application of the rules to the input string.</Paragraph>
      <Paragraph position="10">  Similar kinds of relaxation efforts can be seen in semantic processing. One feature of the preference semantics system of Wilks (1975) is the ability to relax certain semantic constraints. With respect to error identification, Heidorn (1972) dealt with incomplete semantic entities by requesting users to supply missing information based on failures to translate from the internal semantic structures to external computer programs. A somewhat similar process is seen in work by Chang (1978) on the RENDEZ-VOUS system where failure to parse a query leads to a request for clarification from the user.</Paragraph>
      <Paragraph position="11"> With respect to pragmatic errors, Weischedel (1977) introduced a technique which uses presupposition to find certain incorrect uses of words. Joshi and Weischedel (1977) and Weischedel (1979) show that since presuppositions can be computed by a parser and its lexicon they are a class of assumptions inherent in the user input; therefore they can be checked for discrepancies with the system's world knowledge. This work was used and extended by Kaplan (1979) in error identification and recovery in those situations where a user's database query would normally yield only an empty set, i.e. an answer of none. Janas (1979) applied similar techniques to assist the user in the same situations.</Paragraph>
      <Paragraph position="12"> Many of these techniques can be ~pplied to problems of relative ill-formedness. For example, techniques that are being applied in the development of JETS specifically to capture relatively ill-formed sentences will fit within our paradigm (Finin et al., 1979).</Paragraph>
      <Paragraph position="13"> We find the number of techniques that fit within the model we suggest encouraging.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>