File Information
File: 05-lr/acl_arc_1_sum/cleansed_text/xml_by_section/metho/80/j80-2003_metho.xml
Size: 46,417 bytes
Last Modified: 2025-10-06 14:11:16
<?xml version="1.0" standalone="yes"?> <Paper uid="J80-2003"> <Title>Responding Intelligently to Unparsable</Title> <Section position="3" start_page="0" end_page="0" type="metho"> <SectionTitle> 2. Using Presuppositions </SectionTitle> <Paragraph position="0"> Semantic information in a sentence is commonly divided into two classes: given and new information.</Paragraph> <Paragraph position="1"> Given information, or presupposition, is that part of the meaning of a sentence which is presumed true in the context of a discourse. New information is the asserted part. For instance, &quot;The defendant stopped beating his wife&quot;, has as given information that there is some defendant presumed in the context and that that person had been beating his wife. The new information is that the individual ceased that activity.</Paragraph> <Paragraph position="2"> Some presuppositions are associated with the use of syntactic constructs. For instance, all noun phrases making definite reference presume that there is a referent in context. All &quot;wh&quot; questions request new information corresponding to the value of a variable and presuppose the set of constraints on the value of the variable. For instance, &quot;Who is playing the tuba&quot;, presumes that someone is playing the tuba.</Paragraph> <Paragraph position="3"> The meaning of particular words is the source of other examples. The use of certain verbs, such as &quot;describe&quot;, conveys presuppositions, or given information. The question, &quot;What books describe how President Truman died&quot;, has a presupposition that President Truman died. Certain quantifying phrases carry given information, as in &quot;Only project J1 receives parts from New York companies&quot;, which presupposes that project J1 receives parts from New York companies. null An analogy can be drawn between given information and preconditions or &quot;input assertions&quot; on a procedure. Given information for definite noun phrases corresponds to predicates on the value of a variable.</Paragraph> <Paragraph position="4"> Given information from the meaning of predicates such as &quot;describe&quot; corresponds to assertions about the state on entry of a procedure. Therefore, given information includes preconditions on the execution of a user request. Furthermore, such preconditions are directly traceable to particular phrases in that request.</Paragraph> <Paragraph position="5"> The psychological validity of given and new information has been demonstrated by Clark and Haviland (1977) and Haviland and Clark (1974). The psychological process they suggest is that (1) given and new information are first sorted in processing a sentence, (2) memory is then searched to establish that the given information holds in context, and (3) the new information is then asserted in memory.</Paragraph> <Paragraph position="6"> We have modelled this process in natural language systems. Research reported in Joshi and Weischedel (1977) and Weischedel (1979) demonstrated how to organize an augmented transition network and lexicon to compute the given and new information of a sentence. null In another system, we implemented the second of the three parts of the psychological process suggested by Clark and Haviland. That system was an intelligent tutor which pinpointed errors a student makes while answering questions in German during a reading comprehension exercise (Weischedel, et.al., 1978). A text presented to English-speaking students in German provides a relatively closed world for the tutor system, since questions refer to entities presented in the text and facts about them. Therefore, these can be included as a detachable module of world knowledge specific to the particular text, along with any other world knowledge that is applicable to the set of questions. It is still possible for the student to refer to knowledge not contained in the model, but it is unlikely. Though the students have vast amounts of knowledge not in the system model, they have insufficient vocabulary and syntactic forms to be able to express this knowledge initially.</Paragraph> <Paragraph position="7"> Thus, in the environment of foreign language instruction, the system is in the unique position of having more vocabulary and syntactic forms than the user and, therefore, has more domain knowledge than the 98 American Journal of Computational Linguistics, Volume 6, Number 2, April-June 1980 Ralph M. Weischedel and John E. Black Responding Intelligently to Unparsable Inputs user can express. Obviously most systems do not have this property.</Paragraph> <Paragraph position="8"> Presuppositions were very effective in the German tutor system, though they are a crucial semantic check for natural language systems in general. Checking presuppositions against the world model of the German tutor provides recognition for several types of errors. First, given information of questions presented by the system must be inferable from a student's answer; otherwise the answer is inappropriate for the question. Consequently, the tutor diagnoses misunderstanding of a question by checking that the given information of a question (which it knows independently) is among the inferences of a student's answer. Only a very simple inference mechanism is used.</Paragraph> <Paragraph position="9"> Second, given information in the student's answer is checked against the world model. If the given information does not exist in the system's knowledge base, the tutor finds one of two errors. If the presupposition is from a definite noun phrase, the tutor prints the noun phrase and informs the student that it knows of nothing that it could refer to. If the presupposition is associated with the semantics of a particular word, it assumes that this student, who is just learning German, has used the word incorrectly. For instance, essen presupposes that the one eating is human; fressen presupposes that the one eating is an animal.</Paragraph> <Paragraph position="10"> Given information is important for any question-answering system with natural language input. The system must check the presuppositions of the input in order to guarantee that the user's assumptions are true in its world. If any are not, the system can list precisely the assumptions which are not true.</Paragraph> <Paragraph position="11"> These ideas were discussed first in Weischedel (1977) and in Weischedel, et.al. (1978). Kaplan (1977,1979) develops the ideas much further, specifically for data base systems. He postulates a hierarchy for the presuppositions of an English query and has implemented strategies for guiding the user to new queries when the data base would list the empty set in response to a query.</Paragraph> <Paragraph position="12"> Presupposition has received much attention in linguistics and philosophy; see for example Oh and Dineen (1979), Karttunen (1973), and Karttunen and Peters (1975).</Paragraph> <Paragraph position="13"> 3. Two Mechanisms for Diagnosing Syntactic Failures We assume that the purpose of a syntactic component is to translate from natural language input to an internal semantic representation of the input. This need not be a completely syntactic process, but may use semantic computations and contextual expectations to guide the parsing/translating process. Several sources could prevent this process from finding a translation of the input. (We will refer to the input component as a &quot;parser&quot;, though we do not presume that a parse tree is ever explicitly computed.) An important way that an input may fail to parse is when the user employs incorrect forms of the language. If particular forms are anticipated, they may be explicitly included in the syntactic model along with the appropriate translation mechanism. In the German tutor mentioned in the previous section, there are several examples of this. For instance, English-speaking students frequently forget to put past participles at the end of a clause; e.g. using &quot;Ich habe gegessen das Fleisch&quot; rather than the correct &quot;Ich habe das Fleisch gegessen,&quot; (I have eaten the meat). The path in the augmented transition net (ATN) corresponding to the incorrect form computes a message to tell students of the mistake, as well as computing the semantic representation of the answer for semantic analysis. This is particularly effective in the tutor system to catch instances of a student using English syntax patterns rather than German ones.</Paragraph> <Paragraph position="14"> In a similar way, any natural language processing system may include all anticipated forms and translation rules for them whether or not they are strictly proper for the language.</Paragraph> <Paragraph position="15"> Another way for a system to accept incorrect forms of language is suggested by observing a common style of writing grammars. Syntactic input components are often designed using a context-free grammar where each grammar rule may be augmented by predicates operating on the semantic representations or on the constituents linked by the grammar rule. The predicates must be satisfied for the constituents to be grouped as a larger constituent. (Of course, the grammar is no longer context-free then.) Augmented phrase structure grammars (Heidorn, 1975) encode parsers and translators specifically in this way. The augmented transition network formalism also directly lends itself to writing parsers and translators in this way by the predicates on arcs. The version of systemic grammar implemented by Winograd (1972) has this flavor as well. Still another example of this style of writing parsers is the linguistic string parser of Sager (1973) and Grishman (1973).</Paragraph> <Paragraph position="16"> A straightforward example of the use of such predicates is for subject-verb agreement. It is easy for a user to make mistakes in long English sentences, resuiting in parser failure. A solution would be simply to remove the predicate from the rule. However, Grishman (1973) reports from their experience in processing scientific texts that the predicates effectively eliminate a large number of spurious parses.</Paragraph> <Paragraph position="17"> We suggest that, instead of forcing all predicates to be satisfied or ignoring the information inherent in them, that the designer should designate that certain predicates can be relaxed, with a record being kept of each predicate not satisfied during parsing. Only pars-American Journal of Computational Linguistics, Volume 6, Number 2, April-June 1980 99 Ralph M. Weischedel and John E. Black Responding Intelligently to Unparsable Inputs es yielding the fewest unsatisfied predicates are completed. Since the number of predicates that evaluate to false in a partial parse is a non-decreasing number, only those partial parses with the fewest unsatisfied predicates have to be continued. Thus, the number of spurious parses added should be small. (Instead of assuming that all failed predicates have equal weight, one could assign a partial order to them; but we have not yet investigated this.) This strategy was very effective in the German tutor system. Not only were several predicates allowed to fail, but also a procedural specialist was attached to the appropriate arc of the ATN to compute a specific error message and probable cause for the student's error. Subject-verb agreement is one example. Another is noun phrase declension, which is crucial to distinguishing &quot;Das M~idchen gab dem Mann einen Hut&quot; (the girl gave the man a hat) from &quot;Dem M~tdchen gab der Mann einen Hut&quot; (the man gave the girl a hat).</Paragraph> <Paragraph position="18"> The notion of allowing certain predicates to go unsatisfied is much more general than the highly special environment of the German tutor. In the system described in the next section, several predicates were made optional or &quot;failable&quot;. By &quot;failable&quot; we mean that the predicates ought to be true for the pattern to match, but could be false without preventing the pattern from matching if there would be no parse with all such predicates true. In addition to subject-verb agreement, pronominal case was also made failable.</Paragraph> <Paragraph position="19"> The two together allow a sentence such as &quot;Me think him win often&quot; to be parsed, even though the parser has a model of correct language.</Paragraph> </Section> <Section position="4" start_page="0" end_page="0" type="metho"> <SectionTitle> 4. Responses to Unparsable Sentences </SectionTitle> <Paragraph position="0"> Some sentences will not be parsable even using the mechanisms described in the previous section. If one uses an augmented transition network as a top-down, left-right parser, the arcs leaving a state where a parse is blocked offer a set of predictions or expectations regarding what should occur next in the input string.</Paragraph> <Paragraph position="1"> These predictions include more than just the symbols or constituents that were expected to follow; they include the partial interpretation that was being followed. In fact, this partial interpretation is potentially far more informative than the symbols or constituents that were expected next. (In the realm of programming languages, an Algol compiler that gives a syntax error message of &quot;SEMI-COLON EXPECTED&quot; can be quite frustrating since the cause of the problem can be quite difficult to find.) Thus, one of our major goals was to develop and test heuristics that would enable a natural language system to describe what interpretation it was following as an explanation of why it expected specific items which were not present.</Paragraph> <Paragraph position="2"> Our approach is that the parser writer can assign meaning to the states of a parser as it is being written, somewhat analogous to assigning meaning to programs (Floyd, 1967). Floyd suggested postulating computational states between the actions of a program and associating predicates with these states to capture the intent of the computational state. States are explicitly given in an ATN. The designer's insight into the meaning of a particular state offers potentially a wealth of information that can be presented to the user about the interpretation being followed and why it failed. This may be of significant help in selecting an alternative way to express the input.</Paragraph> <Paragraph position="3"> The meaning of an ATN state may be specified by an ordered list of condition-action pairs, encoded as arbitrary LISP functions. These conditions and actions may be functions of the current word in the input, the previous word in the input, any ATN register having a value as of that state, any ATN register from higher levels in the graph, or the sequence of states traversed.</Paragraph> <Paragraph position="4"> These condition-action pairs furnish a natural way to distinguish among several interpretations or paths that are collapsed at a particular state. The conditions are used to unravel the collapsed paths by referring to the ATN registers and input string. The action for any given condition provides a flexible way of computing and printing messages the parser writer has chosen to describe the interpretation being followed.</Paragraph> <Paragraph position="5"> In general, the effectiveness of this idea for generating responses to unparsable sentences will depend on heuristics for determining the state at which the problem in the input was encountered. These ideas should be very effective for natural language front ends to applications such as data base systems, expert consultant systems, and computer-assisted instruction systems. null The ideas do not presume that the parser operates sequentially or prior to semantic or pragmatic components. The ideas would fit in equally well in a multiprocessing environment where syntactic, semantic, and pragmatic components communicate asynchronously, such as GUS (Bobrow, et.al. 1977). In a multiprocessing system, one would have to write the condition-action pairs to use information and decisions from the other components. The only assumption we have made is that the parser is top-down, left-right, and is written in the formalism of the ATN.</Paragraph> <Section position="1" start_page="0" end_page="0" type="sub_section"> <SectionTitle> 4.1 Selecting a State from the Set of Possible States </SectionTitle> <Paragraph position="0"> In essence, when a parse is blocked, one wants the partial parse nearest to being complete. We have chosen to select partial parses that have moved furthest in the input string, or, in other words, those that match the longest initial string. However, there may be several paths and blocked states matching the longest initial string. Furthermore, the parse may have gone 100 American Journal of Computational Linguistics, Volume 6. Number 2, April-June 1980 Ralph M. Weischedel and John E. Black Responding Intelligently to Unparsable Inputs several words beyond the point where the real problem occurred.</Paragraph> <Paragraph position="1"> As a heuristic, states where a block has occurred are selected only if they are on a &quot;longest path&quot; matching the most input words. The &quot;length of a path&quot; is defined to be the number of arcs traversed other than PUSH or JUMP arcs with the universally (vacuously) true test, since those arcs make no test on the input string nor consume any part of it, (The pseudo-arcs POP are counted.) If there are still several states, one can be selected nondeterministically.</Paragraph> <Paragraph position="2"> Given the state S selected by the heuristic above, there is a tree of states which are reachable from S using only a string of PUSH or JUMP arcs with the universally true test. S is the root of such a tree. The meaning of each of the states of the tree may oftentimes be summarized by the parser designer into one brief description characterizing all of them as one conceptual unit (rather than a disjunction of descriptions of each). For states where this seems inappropriate, a special function (LOOKAHEAD) can be added as an action in the meaning of S to call the message generating routine recursively for each state at distance one from S in the tree described above. Using these two ideas we found that selecting the root S for its meaning, while ignoring its descendants, proved satisfactory in our tests.</Paragraph> <Paragraph position="3"> The heuristic for selecting one partial parse and one state along a path for it was implemented in a particular parser, to be described in section 4.2. We tested these ideas by constructing for each state an unparsable input such that the heuristic would select that state. Some states either could not be a blocking point or could be one only by a non-English input, such as, &quot;John forced Mary not.&quot; After eliminating such states, we tested the heuristic on one sentence for each remaining state.</Paragraph> <Paragraph position="4"> For an input that does not parse, there is some maximal initial input string consumed by any of the partial parse paths. Consider the set of states on the partial parse paths such that at each such state the maximal input string has been parsed in reaching that state. (The set can be more than a singleton even if there is only one path, since PUSH, JUMP, and POP arcs do not consume input symbols.) For the 39 example sentences, the average number of states in that set was four.</Paragraph> <Paragraph position="5"> To measure the effectiveness of employing the heuristic of using only states at the end of a &quot;longest&quot; path (where JUMP and PUSH arcs with a universally true test are not counted in the length of the path), we counted the number of &quot;longest&quot; paths for each example. In 34 of the 39 test cases, this heuristic yielded only one state. In each of the five remaining cases, two states were left as the last state of a longest path. As mentioned earlier, when more than one state is left after selecting only states at the end of a longest path, one can be selected nondeterministically. In three of the five test cases where nondeterminism was used, the two states would have produced essentially the same message, and therefore using both states would have added no insight.</Paragraph> <Paragraph position="6"> Thus, in our test the heuristic seemed very effective. Of course, the effectiveness of the heuristic depends in large part on the style in which the parser is written. We describe the parser next.</Paragraph> </Section> <Section position="2" start_page="0" end_page="0" type="sub_section"> <SectionTitle> 4.2 The Parser on which the Ideas were Tested </SectionTitle> <Paragraph position="0"> We tested these ideas on a parser written in 1975 as part of a Ph.D. thesis (Weischedel, 1975). The purpose of the parser was to demonstrate how to compute two special classes of inferences, presuppositions and entailments of English sentences, by using an ATN and a lexicon. Because of its special purpose, the system included many constructions and lexical items of interest for their presuppositional or entailment content. For the same reason, many frequently occurring constructions of English were not implemented, e.g. conjunction reduction, reduced relative clauses, several consecutive prepositional phrases, and adverbials.</Paragraph> <Paragraph position="1"> A subset of constructions were selected from the linguistic analysis of Anderson (1970), which was a basis of defining the lexical classes of the Linguistic String Parser, described in Sager (1973) and Grishman (1973). Anderson's analysis defined lexical classes by the left contexts (subject) in which a predicate (verb or noun) could occur, the right contexts (object) in which a predicate could occur, and the transformations which could act upon these strings of subject, predicate and object. (Note, in this section the word &quot;predicate&quot; refers to part of an English clause, not a boolean test as in Section 3.) Such strings define the parsable sentential forms; the transformations acting upon the strings give further forms. Of course, our ATN parser encoded all surface forms in the graphs.</Paragraph> <Paragraph position="2"> explicitly. The actions on the ATN arcs had the effect of inverting the transformations given by Anderson while moving along an ATN path matching the surface form.</Paragraph> <Paragraph position="3"> Conditions on the arcs were very significant in the style of our parser. For instance, a condition before POPping from the sentential level checks whether the left and right contexts matched for the word or predicate X form a string in the linguistic model of Anderson (1970).</Paragraph> <Paragraph position="4"> In the lexical entries each semantic representation for a word was associated with corresponding lexical classes. Finding the semantic representation of a sentence, therefore, required determining lexical classes for each of the words.</Paragraph> <Paragraph position="5"> American Journal of Computational Linguistics, Volume 6, Number 2, April-June 1980 101 Ralph M. Weischedel and John E. Black Responding Intelligently to Unparsable Inputs The arc condition which checks whether a predicate X occurred in appropriate right and left contexts was not one of the ones we declared to be &quot;failable&quot;, because this condition was necessary and sufficient for determining associated semantic representations.</Paragraph> <Paragraph position="6"> When the condition was not satisfied, the parser did not have in its lexicon a semantic representation for the word X. Consequently, the condition offered a very tight constraint for ascertaining when the parser had no semantic representation corresponding to an interpretation. Maintaining such tight control over what the system could and could not translate is somewhat akin in philosophy to using strongly typed programming languages with much error checking. People do not seem to have such a style in using natural languages; however, it might be a useful engineering principle for natural language systems where accuracy of understanding is crucial.</Paragraph> <Paragraph position="7"> Another consciously applied strategy in designing the parser was to separate into distinct paths many strings which conceivably could have been merged.</Paragraph> <Paragraph position="8"> The criterion for making a distinct path was whether a string which was syntactically differentiable also had a distinct semantic representation. For instance, cleft sentences, such as &quot;It was Mary who won&quot;, could have been incorporated by simply listing left and right contexts for &quot;be&quot; in the lexicon. However, the syntactic form has distinct meaning, namely in the exampie, the presupposition that &quot;Someone won.&quot; Therefore, the parser has a special path for cleft sentences. This aspect of style yielded several relatively long paths with little branching. For such paths, the messages for a blocked parse can pinpoint the interpretation that was being followed and what was expected next. Examples of this are provided in section 4.3.</Paragraph> <Paragraph position="9"> One of the major advantages of testing our ideas on this parser was the fact that there were many ways in which a sentence could fail to parse. The parser was already available, but more important for the assigning of meaning to its states, its designer was readily available. A further reason for selecting this parser was that it did cover a wide range of constructions and was not a toy grammar.</Paragraph> <Paragraph position="10"> In general, we specifically avoided enhancing the grammar to remove limitations. We wanted a full range of tests and example problems to exercise our ideas as thoroughly as possible. However, simple bugs such as erroneous omissions in lexical entries or typographical errors were corrected as they were detected. Also, we did add one action to most arcs to save surface phrases as they were found, for more helpful responses to the user.</Paragraph> <Paragraph position="11"> The major drawback in selecting this parser for experimentation is its original purpose. Although its purpose is very precise, it did not have a natural task domain. Without a task, it seems impossible to make some significant tests, such as giving naive users a specific goal in the domain, then measuring how many trials they require to achieve the goal in the restricted natural language.</Paragraph> </Section> <Section position="3" start_page="0" end_page="0" type="sub_section"> <SectionTitle> 4.3 Examples and Analyses </SectionTitle> <Paragraph position="0"> In this section, we have organized example states, messages, and analyses around several themes; each of the following subsections comprises one theme. All graphs here are much simplified recursive transition net approximations of the actual graphs in Weischedel (1979). A double circle indicates a pop arc. Lower case indicates terminals; upper case indicates nonterminals. null Though the parser writer may know precisely what interpretation was being followed and what caused it to block at a given state, it is very challenging to phrase that knowledge in a way helpful to the user. This is a problem common to all natural language systems, but the degree of the problem varies with the application of the system and with the style of the grammar. For instance, in the environment of an intelligent tutor for computer-assisted language instruction, the user is learning or has learned many informal grammatical concepts of the language (though these may not directly correspond to the formal ones implemented in the parser). Consequently, the parser writer in creating response messages as part of the condition-action pairs can use these concepts to pinpoint for the user the reason the parser blocked. In other applications, the user might have few, if any, concepts of grammar.</Paragraph> <Paragraph position="1"> Since our tests were conducted on the English parser for generating presuppositions and entailments, the response messages were aimed at general users having only a few informal concepts of language, such as sentence, subject, verb, and object. In addition, the responses often include examples with a similar structure, rather than using technical terms. For instance, suppose that the phrase &quot;It was the'class ...&quot; was being interpreted as a cleft sentence when it was blocked. The system prints that the input was being interpreted in the same way as a sentence such as &quot;It is John who left,&quot; rather than calling it a cleft sentence. null The style of the particular parser also has a significant effect on the ability to phrase the reason for a parsing failure. For instance, if one uses a &quot;semantic grammar&quot; (Burton, 1976 and Brown and Burton, 1975) the parser writer can use the concepts of the domain encoded in the grammar as a powerful description of the interpretation being followed and of the cause of a blocked parse. In INLAND (Hendrix, et.al., 1978), one can see how effective the domain concepts encoded in the .semantic grammar can be in responding to the user. One class of response messages in INLAND is a list of the elements that could have occurred next when the parse blocked. Even though this list does not indicate the interpretation being followed, the semantic concepts of the domain which could occur next (e.g. ship) are more meaningful to the user than such a list for a general-purpose grammar would be (e.g. noun phrase).</Paragraph> <Paragraph position="2"> In effect, then, we tested the idea of using the meanings of states to generate responses on the hardest case, where the parser is general and the users are completely naive about even informal grammatical concepts. The remainder of this subsection describes the problems we encountered, by examples.</Paragraph> <Paragraph position="3"> State 02 of Figure 1 exemplifies a very frustrating aspect of devising appropriate descriptions. 02 is part of the subgraph that recognizes right contexts (objects) of predicates. The meaning of the nonterminals is as follows.&quot; NP, a noun phrase; NPS, a possessive form of a noun phrase; S, a declarative sentence; WH-S, a wh-question; P, a preposition; V+ing, a verb's present participle; and V+en, a verb's past participle. Though there are four different possible reasons for a parse to block at 02, each of which is rather reliably recognized by a simple condition, the messages describing the problem are not precise.</Paragraph> <Paragraph position="4"> Four condition-action pairs represent the meaning assigned to 02. The first checks for the input string being empty. If it is, the lexical entry for the predicate does not include the appropriate right context being matched, and therefore has no translation for it.</Paragraph> <Paragraph position="5"> Though the problem is pinpointed, describing it to the user is not easy; examples of uses that the system can understand seem to be the most helpful for this. Example 1 demonstrates the message. Each lexical item corresponding to a predicate has a list of sentences, one for each implemented left-right context pair; the examples are stored on disk files and read only if requested. null</Paragraph> </Section> </Section> <Section position="5" start_page="0" end_page="0" type="metho"> <SectionTitle> WOULD YOU LIKE EXAMPLES ? *YES EXAMPLES FOR THE USAGE OF 'PREVENTED' 'THAT THE STUDENTS DID NOT ATTEND THE LECTURE PREVENTED THE PROFESSOR FROM ASSIGNING THE TEXT /.' 'JOHN PREVENTED MARY FROM ATTENDING THE LECTURE /.' 'JOHN WAS PREVENTED FROM LEAVING BY MARY/.' 'JOHN WAS PREVENTED FROM TRANSLATING THE ASSIGNMENT BY ME /.' </SectionTitle> <Paragraph position="0"> A second condition-action pair associated with 02 is apparently never used, because the parse can always continue beyond state 02. This pair checks two conditions: whether the current input symbol is &quot;that&quot; and whether two noun phrases form an appropriate right American Journal of Computational Linguistics, Volume 6, Number 2, April-June 1980 . 103 Ralph M. Weischedel and John E. Black Responding Intelligently to Unparsable Inputs context for the predicate found. In this case the parser should have interpreted &quot;that&quot; as being used referentially. However, we had not included its referential sense in the lexicon. The message associated with the pair would pinpoint the problem were this the final state of a longest parse. In one instance, the parser can go one arc farther by pushing for a noun phrase and verifying that the predicate allows such a right context. Example 2 is such an instance.</Paragraph> </Section> <Section position="6" start_page="0" end_page="0" type="metho"> <SectionTitle> STUCK AT THE WORD 'THAT' LOOKING FOR A NOUN PHRASE, BUT THE WORD 'THAT' CANNOT BE USED AS A PRONOUN IN THIS SYSTEM TO REFER TO SOMETHING </SectionTitle> <Paragraph position="0"> Far more likely circumstances are that the parser can continue by interpreting &quot;that&quot; as the beginning of a relative clause modifying the noun phrase matched in reaching state 02. The meaning of the last state in that case does not pinpoint the problem, but at least it does explain the interpretation being followed, as demonstrated in Example 3.</Paragraph> <Paragraph position="1"> In Example 3, the parser has gone one word beyond the real difficulty in the input. The problem of going beyond where the real block occurred is more apparent than real for state 02, however. If we had not decided a priori that for the purposes of testing our ideas we would not add to the parser or lexicon, we would have simply added the referential sense of &quot;that&quot; to the lexicon.</Paragraph> <Paragraph position="2"> A third condition-action pair associated with 02 deals with an error in a design decision made when first building the parser. In Anderson (1970), the lexical analysis cites many predicates whose right contexts include prepositions specific to a particular predicate. For instance, &quot;tell&quot; has right contexts specifically allowing &quot;of&quot; or &quot;about&quot;. Paths leaving 02 upon finding a preposition require that it specifically be listed in the lexical entry of the predicate. However, in 1975 we made the erroneous assumption that only one preposition would be listed per predicate. The condition-action pair checks whether this could be the problem; unfortunately, describing the problem to a naive user is itself a problem. As Example 4 indicates, the best description'we could think of is the same as for the first condition-action pair of 02 (Example 1).</Paragraph> <Paragraph position="3"> used if no other pair applies. There are two possible causes in this case: the predicate's lexical entry might not include the right context present in the sentence or the NP that was just matched could have prepositional phrases modifying it. The message is essentially the same as that in Example 1. The cause, like the message, is not precise in this case.</Paragraph> <Paragraph position="4"> State 02 illustrates that even though the designer may assign condition-action pairs that pinpoint the cause for a sentence not being parsed, descriptions of the cause may not be as precise or helpful to a naive user. Thus, the messages can be less helpful than one would have hoped.</Paragraph> <Paragraph position="5"> 104 American Journal of Computational Linguistics, Volume 6, Number 2, April-June 1980</Paragraph> <Paragraph position="7"> In spite of the problem illustrated in the last section, much precision is possible in messages to the user. For example, state S17, which appears in the full diagrams of Weischedel (1979), is on the path for recognizing a subset of the cleft sentences. The path that it is on is an example of many paths that are very long, with little branching, and that correspond to a particular interpretation. (This is a characteristic of the style of the parser.) At S17, the word &quot;it&quot;, a string of tense and modal elements ending in a form of &quot;be&quot;, and a noun phrase have been matched. The only arc leaving S17 matches a relative clause.. If a block occurs here, either the input was not a complete cleft sentence, or the relative clause began in an unparsable way. The message printed appears as Example 5. The portion of the message describing relative clause restrictions was generated from the condition-action pairs of a different state; that state's pairs were involved because S17's pair explicitly called the LOOKAHEAD function after printing the first part of the message.</Paragraph> <Paragraph position="8"> Another example of the kind of precision possible comes from one of the messages of $9, shown in Figure 2. LSUBJ matches left contexts of a predicate; in this case the left context is the surface subject of the verb. TENSE(be) will match any tensed elements ending in a form of &quot;be&quot;. V+en represents a past participle of a verb. POBJ looks for the right context of the verb, thus matching right contexts from which the surface subject was syntactically moved. By the time $9 has been reached, the system is interpreting the input as a passive sentence.</Paragraph> <Paragraph position="9"> The first condition-action pair associated with $9 checks whether the past participle found is in a particular lexical subcategory, because passives of that sub-category are treated in a special manner. The arcs for the special case were not implemented. The printed message appears in Example 6 and corresponds exactly to the omission in the grammar.</Paragraph> <Paragraph position="11"/> </Section> <Section position="7" start_page="0" end_page="0" type="metho"> <SectionTitle> CROWDED /. NO PARSES I WAS DISAPPOINTED STUCK AT THE WORD 'THAT' CURRENT SYSTEM CANNOT HANDLE PASSIVE SENTENCES INVOLVING 'DISAPPOINTED'. </SectionTitle> <Paragraph position="0"> A second condition-action pair for $9 always prints a message if the first one did not apply. This clause corresponds to a general reason for blocking at $9: none of the expected right contexts for the verb could be found. This could arise if the lexical entry did not list the necessary right context and therefore had no translation for this case. It could also arise in a sentence such as &quot;That I won was told immediately to Mary.&quot; (Recall that we simply did not include adverbial adjuncts in the parser.) Just as the cause is not very precise for this instance, the message given in Example 7 cannot be either. The example sentences given as output do parse. The input does not parse because the lexical entry simply did not include a noun phrase as one of its right contexts.</Paragraph> </Section> <Section position="8" start_page="0" end_page="0" type="metho"> <SectionTitle> 'THAT MARY LEFT WAS BELIEVED BY THE STUDENTS /.' </SectionTitle> <Paragraph position="0"> Using states S17 and $9 along with the corresponding Examples 5 and 6, we have demonstrated that the messages can sometimes pinpoint the cause of a parsing failure. There are many other states whose condition-action pairs yield a precise diagnosis for the cause of a parsing failure.</Paragraph> <Paragraph position="1"> For sentences with embeddings, merely to give information based on the last state of the longest path seems intuitively insufficient, for explanation of the higher levels of the sentence may be ignored if the message is based solely on the last state at an embedded level. Consequently, the system prints messages for each incomplete sentential level represented in the partial parse. First, the message from the last state is printed. Then, starting at the highest level, an explanatory message is printed for each incomplete sentential level.</Paragraph> <Paragraph position="2"> These messages are printed using the same ideas as described for the last state on the longest path. The criterion for selecting states is simple. The parser's stack contains all the states with an exiting PUSH arc that has been started but remains unfinished. Of the states in that stack, only the ones corresponding to a sentential level are relevant; these begin with an &quot;S&quot; or an &quot;I&quot; in our graph. The set of condition-action pairs for these states was written assuming this was the last state on the longest path. Consequently, we wrote a second, smaller set of condition-action pairs especially assuming that partially parsed embedded sentences follow this state.</Paragraph> <Paragraph position="3"> Example 8 illustrates messages for embedded sentences. The output beginning with &quot;This error occurred while ...&quot; is the start of messages from higher level, partially parsed sentences. The useful hint at the true problem in parsing Example 8 comes from one of the states in the system's stack; the right context necessary to parse Example 8 has not been defined.</Paragraph> </Section> <Section position="9" start_page="0" end_page="0" type="metho"> <SectionTitle> INTERPRETATION OF THE WAY IT EXPECTED THAT EMBEDDED SENTENCE TO FIT INTO THE COMPLETE SENTENCE WAS: SUBJECT UNDERSTOOD TO BE: 'MARY' VERB UNDERSTOOD TO BE: 'DID ASK' LOOKING FOR AN APPROPRIATE OBJECT FOR </SectionTitle> <Paragraph position="0"> 'ASK'.</Paragraph> <Paragraph position="1"> 4.3.4 Testing the Longest Path Heuristic A serious difficulty in using the longest path as a heuristic for generating responses is that the parser may be able to continue further in the input than where the real parsing problem occurred. To examine how well the longest path heuristic performs in locating the true cause of the problem, we analyzed the 39 sentences described in section 4.1. In only three of the 39 cases did the parser continue beyond the point where the true problem occurred. Contrasted with this success rate, Woods (personal communication, 1977) reported that in LUNAR, the parser very often was able to continue beyond the point of the problem in the input before becoming blocked.</Paragraph> <Paragraph position="2"> 106 American Journal of Computational Linguistics, Volume 6, Number 2, April-June 1980 Ralph M. Weischedel and John E. Black Responding Intelligently to Unparsable Inputs There are several factors that affect the success of the longest path heuristic. One is the extent of the grammar; the fact that adverbial adjuncts, reduced relative clauses, and multiple, consecutive prepositional phrases are not present in the grammar we tested undoubtedly contributed to the high success rate. Therefore, the heuristic should be very effective in applied natural language interfaces that are constrained.</Paragraph> <Paragraph position="3"> Second, the style of grammar can affect the success of the heuristic. For instance, our grammar immediately upon finding the main predicate (e.g. verb) of a clause requires that its syntactic expectations for right contexts of that particular main predicate be satisfied at each step through the remainder of the string containing a right context. Also, as near as possible, semantically different senses were usually separated into distinct paths, even though they might have been collapsed into one.</Paragraph> <Paragraph position="4"> Third, applying semantic constraints and expectations while parsing should also contribute to the effectiveness of the longest path heuristics, just as the syntactic constraints and expectations do. The additional constraints will inhibit the parser from continuing beyond a problem in the input by preventing it from processing a phrase with the expected syntactic form but which is unacceptable semantically. For instance, suppose the actual right context of a predicate (e.g.</Paragraph> <Paragraph position="5"> verb) starts with a noun phrase, but the lexicon lists no right contexts for the predicate that begin with a noun phrase. A parser might be able to continue by interpreting the noun phrase as an adverbial adjunct specifying a time, such as &quot;last night.&quot; If the parser interacts with a semantic component requiring that the noun phrase be interpretable as a time specification, the parser could not go on by interpreting the noun phrase erroneously. Since our grammar does not interact with a semantic component, we are interested in testing the longest path heuristic in RUS (Bobrow, 1978), a grammar which does interact closely with semantics.</Paragraph> <Paragraph position="6"> A natural criterion for evaluating this strategy for unparsable sentences is the cost, both in processing and programming development. In processing, very little is added. Clearly, a small fraction of the parsing time and memory usage is added to record the longest path and to generate messages for the last state on it (and possibly one state per incomplete sentential level). However, it is easy to see that this is a minute fraction compared to the time and memory in searching for a parse.</Paragraph> <Paragraph position="7"> On the other hand, significant additional effort is required of the programmer to devise condition-action pairs for each state. However, spending that time has benefits in addition to the response ability added to the system. Analyzing the parser to develop the meaning of each state clarifies the programmer's understanding of the system. Furthermore, it serves as significant documentation, since it describes the intent of the programmer at each point.</Paragraph> <Paragraph position="8"> For our graph having approximately 110 states, the average number of condition-action pairs per state was 1.4. The code for these pairs amounted to approximately one page of a listing for the conditions and approximately nine pages for the constant character strings used in generating the (rather long) printed messages. Therefore, it is clear that the condition-action pairs do not require a lot of programming, but do require a better understanding and description of the parser.</Paragraph> </Section> class="xml-element"></Paper>