<?xml version="1.0" standalone="yes"?>
<Paper uid="P94-1030">
  <Title>GENERALIZED CHART ALGORITHM: AN EFFICIENT PROCEDURE FOR COST-BASED ABDUCTION</Title>
  <Section position="2" start_page="0" end_page="223" type="concl">
    <SectionTitle>
Abstract
</SectionTitle>
    <Paragraph position="0"> We present an efficient procedure for cost-based abduction, which is based on the idea of using chart parsers as proof procedures. We discuss in detail three features of our algorithm -- goal-driven bottom-up derivation, tabulation of the partial results, and agenda control mechanism -- and report the results of the preliminary experiments, which show how these features improve the computational efficiency of cost-based abduction.</Paragraph>
    <Paragraph position="1"> Introduction Spoken language understanding is one of the most challenging research areas in natural language processing. Since spoken language is incomplete in various ways, i.e., containing speech errors, ellipsis, metonymy, etc., spoken language understanding systems should have the ability to process incomplete inputs by hypothesizing the underlying information. The abduction-based approach (Hobbs et al., 1988) has provided a simple and elegant way to realize such a task.</Paragraph>
    <Paragraph position="2"> Consider the following 3apanese sentence:  (1) Sfseki kat- ta  (a famous writer) buy PAST This sentence contains two typical phenomena arising in spoken language, i.e., metonymy and the ellipsis of a particle. When this sentence is uttered under the situation where the speaker reports his experience, its natural interpretation is the speaker bought a SSseki novel. To derive this interpretation, we need to resolve the following problems:  verb katta and the noun phrase SSseki is determined to be the object case relation, based on the semantic knowledge that the object case relation between a trading action and a commodity can be linguistically expressed as a thematic relation. This interpretation is made by abduction. For instance, the above semantic knowledge is stated, in terms of the predicate logic, as follows: (2) sem(e,x) C trade(e) A commodity(x) A obj(e,x) Then, the inference process derives the consequent sem(e,x) by hypothesizing an antecedent obj(e,x), which is never proved from the observed facts. This process is called abduction.</Paragraph>
    <Paragraph position="3"> Of course, there may be several other possibilities that support the thematic relation sem(e,x). For instance, the thematic relation being determined to be the agent case relation, sentence (1) can have another interpretation, i.e., Sfseki bought something, which, under some other situations, might be more feasible than the first interpretation. To cope with feasibility, the abduction-based model usually manages the mechanism for evaluating the goodness of the interpretation. This is known as cost-based abduction (Hobbs et al., 1988).</Paragraph>
    <Paragraph position="4"> In cost-based abduction, each assumption bears a certain cost. For instance, the assumption obj(e,x), introduced by applying rule (2), is specified to have a cost of, say, $2. The goodness of the interpretation is evaluated by accumulating the costs of all the assumptions involved. The whole process of interpreting an utterance is depicted in the following schema:  1. Find all possible interpretations, and 2. Select the one that has the lowest cost.</Paragraph>
    <Paragraph position="5">  In our example, the interpretation that assumes the thematic relation to be the object case relation, with the metonymy being expanded to a S6seki novel, is cheaper than the interpretation that assumes the thematic relation to be the agent case relation; hence, the former is selected.</Paragraph>
    <Paragraph position="6"> An apparent problem here is the high computational cost; because abduction allows many possibilities, the schema involves very heavy computation. Particularly in the spoken language understanding task, we need to consider a great number of possibilities when hypothesizing various underlying information. This makes the abduction process  computationally demanding, and reduces the practicality of abduction-based systems. The existing models do not provide any basic solution to this problem. Charniak (Charniak and Husain, 1991; Charniak and Santos Jr., 1992) dealt with the problem, but those solutions are applicable only to the propositional case, where the search space is represented as a directed graph over ground formulas. In other words, they did not provide a way to build such graphs from rules, which, in general, contain variables and can be recursive.</Paragraph>
    <Paragraph position="7"> This paper provides a basic and practical solution to the computation problem of cost-based abduction. The basic idea comes from the natural language parsing literature. As Pereira and Warren (1983) pointed out, there is a strong connection between parsing and deduction. They showed that parsing of DCG can be seen as a special case of deduction of Horn clauses; conversely, deduction can be seen as a generalization of parsing. Their idea of using chart parsers as deductive-proof procedures can easily be extended to the idea of using chart parsers as abductive-proof procedures. Because chart parsers have many advantages from the viewpoint of computational efficiency, chart-based abductive-proof procedures are expected to nicely solve the computation problem. Our algorithm, proposed in this paper, has the following features, which considerably enhance the computational efficiency of cost-based abduction:  1. Goal-driven bottom-up derivation, which reduces the search space.</Paragraph>
    <Paragraph position="8"> 2. Tabulation of the partial results, which avoids the recomputation of the same goal.</Paragraph>
    <Paragraph position="9"> 3. Agenda control mechanism, which realizes var- null ious search strategies to find the best solution efficiently.</Paragraph>
    <Paragraph position="10"> The rest of the paper is organized as follows.</Paragraph>
    <Paragraph position="11"> First, we explain the basic idea of our algorithm, and then present the details of the algorithm along with simple examples. Next, we report the results of the preliminary experiments, which clearly show how the above features of our algorithm improve the computational efficiency. Then, we compare our algorithm with Pereira and Warren's algorithm, and finally conclude the paper.</Paragraph>
    <Section position="1" start_page="218" end_page="220" type="sub_section">
      <SectionTitle>
Head-driven Derivation
</SectionTitle>
      <Paragraph position="0"> Pereira and Warren showed that chart parsers can be used as proof procedures; they presented the Earley deduction proof procedure, that is a generalization of top-down chart parsers. However, they mentioned only top-down chart parsers, which is not always very efficient compared to bottom-up (left-corner) chart parsers. It seems that using left-corner parsers as proof procedures is not so easy,  unless the rules given to the provers have a certain property. Here, we describe under what conditions left-corner parsers can be used as proof procedures. Let us begin with the general problems of Horn clause deduction with naive top-down and bottom-up derivations: * Deduction with top-down derivation is affected by the frequent backtracking necessitated by the inadequate selection of rules to be applied.</Paragraph>
      <Paragraph position="1"> * Deduction with bottom-up derivation is affected by the extensive vacuous computation, which never contributes to the proof of the initial goal. These are similar to the problems that typically arise in natural language parsing with naive top-down and bottom-up parsers. In natural language parsing, these problems are resolved by introducing a more sophisticated derivation mechanism, i.e., left-corner parsing. We have attempted to apply such a sophisticated mechanism to deduction.</Paragraph>
      <Paragraph position="2"> Suppose that the proof of a goal g(x,y) can be represented in the manner in Figure 1; the first argument x of the goal g(x,y) is shared by all the formulas along the path from the goal g(z,y) to the left corner am(z,zm). In such a case, we can think of a derivation process that is similar to left-corner parsing. We call this derivation head-driven derivation, which is depicted as follows: Step 1 Find a fact a(w,z) whose first argument w unifies with the first argument x of the goal g(x,y), and place it on the left corner.</Paragraph>
      <Paragraph position="3"> Step 2 Find a rule am-l(W,Zrn-l) C a(W,Zm)/~ BZ ^ ... A Bn whose leftmost antecedent a(W,Zm) unifies with the left-corner key a(x,z), and introduce the new goals B1, ..., and Bn. If all these goals are recursively derived, then create the consequent a,,~_ 1 ( z ,zm_ 1 ), which dominates a(x,zm), B1, ..., and Bn, and place it on the left corner instead of a(x,z).</Paragraph>
      <Paragraph position="4"> Step3 If the consequent am-l(x,zm_l) unifies with the goal g(z,y), then finish the process. Otherwise, go back to step2 with am- 1 (x,zm_l) being the new left-corner key.</Paragraph>
      <Paragraph position="5">  Left-corner parsing of DCG is just a special case of head-driven derivation, in which the input string is shared along the left border, i.e., the path from a nonterminal to the leftmost word in the string that is dominated by that nonterminal.</Paragraph>
      <Paragraph position="6"> Also, semantic-head-driven generation (Shieber el al., 1989; van Noord, 1990) and head-corner parsing ivan Noord, 1991; Sikkel and op den Akker, 1993) can be seen as head-driven derivation, when the semantic-head/syntactic-head is moved to the leftmost position in the body of each rule and the argument representing the semantic-feature/headfeature is moved to the first position in the argument list of each formula.</Paragraph>
      <Paragraph position="7"> To apply the above procedures, all rules must be in chain form arn--l(W,Zrn-~) C arn(W,Zm) A B1 A ... A Bn; that is, in every rule, the first argument of the leftmost antecedent must be equal to the first argument of the consequent. This is the condition under which left-corner parsers can be used as proof procedures. Because this condition is overly restrictive, we extend the procedures so that they allow non-chain rules, i.e., rules not in chain form. Step 1 is replaced by the following: Step 1 Find a non-chain rule a(w,z) C B1 A... A B~ such that the first argument w of the consequent a(w,z) unifies with the first argument z of the goal g(x,y), and introduce the new goals B1, ..., and /3,. A fact is regarded as a non-chain rule with an empty antecedent. If all these goals are recursively derived, then create the consequent a(z,z), which dominates B1, ..., and B,, and place it on the left corner.</Paragraph>
      <Paragraph position="8">  The idea given in the previous section realizes the goal-driven bottom-up derivation, which is the first feature of our algorithm. Then, we present a more refined algorithm based upon the idea, which realizes the other two features as well as the first one. Chart Parsing and its Generalization Like left-corner parsing, which has the drawback of repeatedly recomputing partial results, head-driven derivation will face the same problem when it is executed in a depth-first manner with backtracking. In the case of left-corner parsing, the problem is resolved by using the tabulation method, known as chart parsing (Kay, 1980). A recent study by Haruno et al. (1993) has shown that the same method is applicable to semantic-head-driven generation. The method is also applicable to head-driven derivation, which is more general than semantic-head-driven generation.</Paragraph>
      <Paragraph position="9"> To generalize charts to use in proof procedures,</Paragraph>
      <Paragraph position="11"> we first define the chart lexicons. In chart parsing, lexicons are the words in the input string, each of which is used as the index for a subset of the edges in the chart; each edge incident from (the start-point of) lexicon w represents the sub-structure dominating the sub-string starting from w. In our case, from the-similarity between left-corner parsing and head-driven derivation, lexicons are the terms that occur in the first argument position of any formula; each edge incident from (the start-point of) lexicon x represents the substructure dominating the successive sequence of the derived formulas starting from the fact in which z occupies the first argument position. For example, in the chart representing the proof in Figure 1, all the edges corresponding to the formulas on the left border, i.e. am(X,Zrn), am--l(Z,Zm--1),..., al(x,zl) and g(z,y), are incident from (the start-point of) lexicon z, and, hence, x is the index for these edges.</Paragraph>
      <Paragraph position="12"> Following this definition of the chart lexicons, there are two major differences between chart parsing and proof procedures, which Haruno also showed to be the differences between chart parsing and semantic-head-driven generation.</Paragraph>
      <Paragraph position="13">  1. In contrast to chart parsing, where lexicons are determined immediately upon input, in proof procedures lexicons should be incrementally introduced. null 2. In contrast to chart parsing, where lexicons are  connected one by one in a linear sequence, in proof procedures lexicons should be connected in many-to-many fashion.</Paragraph>
      <Paragraph position="14"> In proof procedures, the chart lexicons are not determined at the beginning of the proof (because  we don't know which formulas are actually used in the proof), rather they are dynamically extracted from the subgoals as the process goes. In addition, if the rules are nondeterministic, it sometimes happens that there are introduced, from one left-corner key, a(x,z), two or more distinct successive subgoals, bl(wl,y~), b2(w2,y2), etc., that have different first arguments, w 1, w 2, etc. In such a case, one lexicon x should be connected to two or more distinct lexicons, w 1, w 2, etc. Furthermore, it can happen that two or more distinct left-corner keys, al(xl,zl), a2(x2,z2), etc., incidentally introduce the successive subgoals, bl(w,yl), b2(w,y~), etc., with the same first argument w. In such a case, two or more distinct lexicons, x 1, x 2, etc., should be connected to one lexicon w. Therefore, the connections among lexicons should be manyto-many. Figure 2 shows an example of charts with many-to-many connections, where the connections are represented by pointers A, B; etc.</Paragraph>
      <Paragraph position="15"> The Algorithm We, so far, have considered deduction but not abduction. Here, we extend our idea to apply to abduction, and present the definition of the algorithm. The extension for abduction is very simple.</Paragraph>
      <Paragraph position="16"> First, we add a new procedure, which introduces an assumption G for a given goal G. An assumption is treated as if it were a fact. This means that an assumption, as well as a fact, is represented as a passive edge in terms of the chart algorithm. Second, we associate a set S of assumptions with each edge e in the chart; S consists of all the assumptions that are contained in the completed part of the (partial) proof represented by the edge e. More formally, the assumption set 5 associated with an edge e is determined as follows:  1. If e is a passive edge representing an assumption A, then S-- {A}.</Paragraph>
      <Paragraph position="17"> 2. If e is a passive/active edge introduced from a non-chain rule, including fact, then S is empty.</Paragraph>
      <Paragraph position="18"> 3. If e is a passive/active edge predicted from a chain rule with a passive edge e' being the left-corner key, then S is equal to the assumption set S' of e'.</Paragraph>
      <Paragraph position="19"> 4. If e is a passive/active edge created by combining  an active edge el and a passive edge e2, then ,-q = $1 U $2 where 81 and ~q2 are the assumption sets of el and e2, respectively.</Paragraph>
      <Paragraph position="20"> Taking these into account, the definition of our algorithm is as follows, f is a function that assigns a unique vertex to each chart lexicon. The notation A:S stands for the label of an edge e, where A is the label of e in an ordinary sense and S is the assumption set associated with e.</Paragraph>
      <Paragraph position="21"> Initialization Add an active edge \[\[?IG\]-I-:C/ to the chart, looping at vertex 0, where G is the initial goal.</Paragraph>
      <Paragraph position="22"> Apply the following procedures repeatedly until no procedures are applicable.</Paragraph>
      <Paragraph position="23"> Introduction Let e be an active edge labeled \[...\[?\]Bj...\]A:S incident from vertex s to t, where Bj = bj (zj,yj) is the first open box in e.  1. If the lexicon xj is never introduced in the chart, then introduce it and run a pointer from t to f(zj). Then, do the following: (a) For every non-chain rule a(w,z)C B1 A ... A Bn, including fact, such that w unifies with zi, create an active edge labeled \[\[?\]Bl'&amp;quot;\[?lS,~\]a(xj,z):C/ between vertex f(xj) and f(zj) + 1. (Create, instead, a passive edge labeled a(xj,z):C/ when the rule is a fact, i.e. n = 0.) (b) Create a passive edge labeled Bj:{Bj} between vertex f(xj) and f(zj) + 1.</Paragraph>
      <Paragraph position="24"> 2. If the lexicon zj was previously introduced in  the chart, then run a pointer from t to f(xj).</Paragraph>
      <Paragraph position="25"> In addition, if the passive edge Bj :{Bj } never exists in the chart, create it between vertex f(rj) and f(xj) + 1.</Paragraph>
      <Paragraph position="26"> Prediction Let e be a passive edge labeled C:S incident from vertex s to t. For every chain rule A' C A A B1 A ... A Bn such that A unifies with C, create an active edge labeled \[A\[?\]B1...\[?\]Bn\]A':,~ between vertex s and t. (Create, instead, a passive edge labeled A':S when A is the single antecedent, i.e., n = 0.) Combination Let ez be an active edge labeled \['&amp;quot; &amp;quot;\[?\]Bj\[?\]Bj+I'&amp;quot; .\[?\]B,~\]A:$1 incident from vertex s to t, where Bj is the first open box in ez and let e2 be a passive edge labeled C:S~ incident from vertex u to v. If Bj and C unify and there is a pointer from t to u, then create an active edge labeled \[-..Bj\[?\]Bj+I...\[?\]Bn\]A:S1 US2 between vertex s and v. (Create, instead, a passive edge labeled A:S1 U S: when B 1 is the last element, i.e., j = n.) Each passive edge T:S represents an answer.</Paragraph>
    </Section>
    <Section position="2" start_page="220" end_page="221" type="sub_section">
      <SectionTitle>
Examples
</SectionTitle>
      <Paragraph position="0"> Here, we present a simple example of the application of our algorithm to spoken language understanding. Figure 3 provides the rules for spoken Japanese understanding, with which the sentence (1) is parsed and interpreted. They include the pragmatic, semantic and knowledge rules as well as the syntactic and lexical rules.</Paragraph>
      <Paragraph position="1"> The syntactic rules allow the connection between a verb and a noun phrase with or with-</Paragraph>
      <Paragraph position="3"> out a particle, which permit structures like \[VP\[NpS6sek2\]\[vpkatla\]\]. Such a structure is evaluated by the pragmatic and semantic criteria. That is, the dependency between a verbal concept e and a nominal concept x is supported if there is an entity y such that x and y have a pragmatic relation, i.e., a metonymy relation, and e and y have a semantic relation, i.e., a thematic relation. The metonymy relation is defined by the pragmatic rules, based on certain knowledge, such as that the name of a writer is sometimes used to refer to his novel. Also, the thematic relation is defined by the semantic rules, based on certain knowledge, such as that the object case relation between a trading action and a commodity can be linguistically expressed as a thematic relation.</Paragraph>
      <Paragraph position="4"> The subscript $c of a formula A represents the cost of assuming formula A. A is easy to assume when c is small, while A is difficult to assume when c is large. For instance, the cost of interpreting the thematic relation between a trading action and a commodity as the object case relation is low, say $2, while the cost of interpreting the thematic relation between an intentional action and a third person as the agent case relation is high, say $20. This assignment of costs is suitable for a situation in which the speaker reports his experience. In spite of the difficulty of assigning suitable costs in general, the cost-based interpretation is valuable, because it provides a uniform criteria for syntax, semantics and pragmatics. Hopefully, several techniques, independently developed in these areas, e.g., stochastic parsing, example-based/corpus-based techniques for word sense/structural disambiguation, etc., will be usable for better cost assignment. Probability will also be a key technique for the cost assignment (Charniak and Shimony, 1990).</Paragraph>
      <Paragraph position="5"> Figure 4 and Table 1 show the chart that is created when a sentence (1) is parsed and interpreted using our algorithm. Although the diagram seems complicated, it is easy to understand if we break down the diagram. Included are the syntactic parsing of the sentence (indicated by edges 2, 6, 7, 14, 52 and 53), the pragmatic interpretation of the metonymy by S6seki S (indicated by edges 17, 18, 20 and 24), the semantic interpretation of the thematic relation between a buying event B and a novel N written by S6seki (indicated by edges 42, 44, 45, 47, 48 and 50), and so on. In the pragmatic interpretation, assumption novel(N) (edge 21) is introduced, which is reused in the semantic interpretation. In other words, a single assumption is used more than once. Such a tricky job is naturally realized by the nature of the chart algorithm.</Paragraph>
    </Section>
    <Section position="3" start_page="221" end_page="222" type="sub_section">
      <SectionTitle>
Agenda Control
</SectionTitle>
      <Paragraph position="0"> Since the aim of cost-based abduction is to find out the best solution, not all solutions, it is reasonable to consider combining heuristic search strategies with our algorithm to find the best solution efficiently. Our algorithm facilitates such an extension by using the agenda control mechanism, which is broadly used in advanced chart parsing systems.</Paragraph>
      <Paragraph position="1"> The agenda is a storage for the edges created by any of the three procedures of the chart algorithm, out of which edges to be added to the chart are selected, one by one, by a certain criterion. The simplest strategy is to select the edge which has the minimal cost at that time, i.e., ordered search.</Paragraph>
      <Paragraph position="2"> Although ordered search guarantees that the first solution is the best one, it is not always very efficient. We can think of other search strategies, like best first search, beam search, etc., which are more practical than ordered search. To date, we have not investigated any of these practical search strategies.</Paragraph>
      <Paragraph position="3"> However, it is apparent that our chart algorithm, together with the agenda control mechanism, will provide a good way to realize these practical search strategies.</Paragraph>
    </Section>
    <Section position="4" start_page="222" end_page="223" type="sub_section">
      <SectionTitle>
Preliminary Experiments
</SectionTitle>
      <Paragraph position="0"> We conducted preliminary experiments to compare four methods of cost-based abduction: top-down algorithm (TD), head-driven algorithm (HD), generalized chart algorithm with full-search (GCF), and generalized chart algorithm with ordered search (GCO). The rules used for the experiments are in the spoken language understanding task, and they are rather small (51 chain rules + 35 non-chain rules). The test sentences include one verb and 1-4 noun phrases, e.g., sentence (1).</Paragraph>
      <Paragraph position="1"> Table 2 shows the results. The performance of each method is measured by the number of computation steps, i.e., the number of derivation steps in TD and HD, and the number of passive and active edges in GCF and GCO. The decimals in parentheses show the ratio of the performance of each method to the performance of TD. The table clearly shows how the three features of our algorithm improve the computational efficiency. The improvement from TD to HD is due to the first feature, i.e., goal-driven bottom-up derivation, which eliminates about 50% of the computation steps; the improvement from HD to GCF is due to the second feature, i.e., tabulation of the partial results,  which decreases the number of steps another 13%23%; the improvement from GCF to GCO is due to the last feature, i.e., the agenda control mechanism, which decreases the number of steps another 4%8%. In short, the efficiency is improved, maximally, about four times.</Paragraph>
      <Paragraph position="2"> Comparison with Earley Deduction We describe, here, some differences between our algorithm and Earley deduction presented by Pereira and Warren. First, as we mentioned before, our algorithm is mainly based on bottom-up (left-corner) derivation rather than top-down derivation, that Earley deduction is based on. Our experiments showed the superiority of this approach in our par- null titular, though not farfetched, example.</Paragraph>
      <Paragraph position="3"> Second, our algorithm does not use subsumption-checking of edges, which causes a serious computation problem in Earley deduction. Our algorithm needs subsumption-checking only when a new edge is introduced by the combination procedure. In the parsing of augmented grammars, even when two edges have the same nonterminal symbol, they are different in the annotated structures associated with those edges, e.g., feature structures; in such a case, we cannot use one edge in place of another. Likewise, in our algorithm, edges are always annotated by the assumption sets, which, in most cases, prevent those edges from being reused.</Paragraph>
      <Paragraph position="4"> Therefore, in this case, subsumption-checking is not effective. In our algorithm, reuse of edges only becomes possible when a new edge is introduced by the introduction procedure. However, this is done only by adding a pointer to the edge to be reused, and, to invoke this operation, equality-checking of lexicons, not edges, is sufficient.</Paragraph>
      <Paragraph position="5"> Finally, our algorithm has a stronger connection with chart parsing than Earley deduction does. Pereira and Warren noted that the indexing of formulas is just an implementation technique to increase efficiency. However, indexing plays a considerable role in chart parsing, and how to index formulas in the case of proof procedures is not so obvious. In our algorithm, from the consideration of head-driven derivation, the index of a formula is determined to be the first argument of that formula. All formulas with the same index are derived the first time that index is introduced in the chart. Pointers among lexicons are also helpful in avoiding nonproductive attempts at applying the combination procedure. All the devices that were originally used in chart parsers in a restricted way are included in the formalism, not in the implementation, of our algorithm.</Paragraph>
      <Paragraph position="6"> Concluding Remarks In this paper, we provided a basic and practical solution to the computation problem of cost-based abduction. We explained the basic concept of our algorithm and presented the details of the algorithm along with simple examples. We also showed how our algorithm improves computational efficiency on the basis of the results of the preliminary experiments.</Paragraph>
      <Paragraph position="7"> We are now developing an abduction-based spoken language understanding system using our algorithm. The main problem is how to find a good search strategy that can be implemented with the agenda control mechanism. We are investigating this issue using both theoretical and empirical approaches. We hope to report good results along these lines in the future.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>