<?xml version="1.0" standalone="yes"?>
<Paper uid="C82-1040">
  <Title>A Parser Which Learns the Application Order of Rewriting Rules</Title>
  <Section position="2" start_page="0" end_page="253" type="metho">
    <SectionTitle>
2. Method of Analysis
</SectionTitle>
    <Paragraph position="0"> The input data for this parser is assumed as a word sequence which is the output of a morphological analysis. The output from this parser is a tree structure. The analysis is controlled by the best-first graph-searching technique about the rule applications.</Paragraph>
    <Section position="1" start_page="0" end_page="253" type="sub_section">
      <SectionTitle>
2.1. Description of Rewriting Rules
</SectionTitle>
      <Paragraph position="0"> The rewriting rules transform a list of trees into a list of trees. An example of the rewriting rule in this parser is shown in Fig. 1. It shows that if there is a symbol sequence composed of a tree not-V(erb), a tree N(oun) P(hrase), a tree C(ase-particle), and a tree not-A(dverbial)-P(article) in this order, this is transformed into a tree NP-C.</Paragraph>
      <Paragraph position="1">  Fig. 1 An example of the rewriting rule The right side of rewriting rules is a matching pattern which is to be found in the given input symbol string. Table 1 shows the function symbols to describe the matching patterns. By using these function symbols, it is possible to specify the repetition of pattern elements, to assign data to a variable, and so on. Tt is also  254 M. NAGAO and .L NAKAMURA possible to check the input data by using user-defined functions. These functions enable us to describe complex syntactic functions, semantic relations, and many heuristic checks.</Paragraph>
      <Paragraph position="3"> I Function Match an arbitrary tree* Hatch any number of arbitrary trees.</Paragraph>
      <Paragraph position="4"> an) Evaluate function= fn whose arguments are a Corresponding tree, al, *.., and an. When the Value is not NIL, matching succeed.</Paragraph>
      <Paragraph position="5"> xn) Match any number o4 lists of matching patterns xl * .. xn. Trees are assigned to variable at. (%A xl ... xn) Matching succeeds if all xl, ..., xn are matched to a tree.</Paragraph>
      <Paragraph position="7"> of xl, .*., xn is matched to a tree.</Paragraph>
      <Paragraph position="8"> Matching succeeds if x is not matched to a tree. Table 2 Function symbols of the creation patterns Form Function at If at is a variable, then its value, otherwise at itsel~o (*F fn xl .. xn) The value Of the function: fn whose arguments are xl, * --t xn. (*S at x) The value of a generation Of x assigned to the variable atdeg The left side of rewriting rules is a creation pattern of new syntactic structures. Table 2 shows the function symbols for structure creation. User-defined functions can also be used to check certain relations in this creation pattern. We can generate an arbitrary tree structure by this re-writing rule system.</Paragraph>
      <Paragraph position="10"> Fig. 2 An example of generating three left sides from a right side As shown in Fig. 2, we can specify arbitrary numbers of structures in the left side for the same right side in a rewriting rule.</Paragraph>
      <Paragraph position="11"> Each rewriting rule has a weight {basic score) and a function (fittedness function). The basic score is a static weighting measure which ~eflects the importance of a rule co~pared to the other ruleG of the same category. The basic score is adjusted by a learning process which will be explained in section 3. The 'fittedness function gives a dynamic weighting measure which expresses the fittedness of the rule application to a sentential structure. The function is a user-defined one which can use the data in both the right side and the left sides of the rewriting rules.</Paragraph>
      <Paragraph position="12"> The basic score and the fittedness function are used for the sequence control of rule applications in the best-first graph-searching, which is the essential strategy to get the most plausible structural analysis first.</Paragraph>
      <Paragraph position="13"> 2.2. Flow of Analysis Fig. 3 shows an intermediate structure in the cOurse of a sentence analysis.</Paragraph>
      <Paragraph position="15"> Fig. 3 The structure of a sentence during the analysis {LOT) This structure is represented by a list of trees. We call this structure as a LOT {List of Trees). An analysis step is an application of a rewriting rule to a LOT as shown in Fig. 4. which changes the content of the LOT.</Paragraph>
      <Paragraph position="17"/>
    </Section>
  </Section>
  <Section position="3" start_page="253" end_page="253" type="metho">
    <SectionTitle>
PARSER LEARNING ORDER OF REWRITING RULES 255
</SectionTitle>
    <Paragraph position="0"> To obtain the result of an analysis one by one in the order of plausibility, we use the best-first graph-searchlng technique. If we regard a LOT as a node in a search graph, the new LOT created by the application of a rewriting rule to an old LOT is a sistee node. When several rules are applicable to a LOT or the rule has several left sides, the same number of sister nodes are created from one mother node. The progress of analysis can be represented by an expansion tree (in general, by a graph) as shown in Fig.</Paragraph>
    <Paragraph position="2"> Fig. 5 Search tree This tree can be regarded as a search tree. We expand the node which has the highest evaluation value (the score assigned to the LOT} first. The expansion is the application of a re-writing rule to a LOT. The evaluation value is obtained by the summation of the following four values:  (1) the evaluation value of the mother node.</Paragraph>
    <Paragraph position="3"> (2 the basic score which is attached to the applied rule.</Paragraph>
    <Paragraph position="4"> (3) he value obtained from the t ittedness function which ~ is attached to the applied rule.</Paragraph>
    <Paragraph position="5"> (4) the score of the sentential pattern (SP. which will be explained in section 2.5), if it matches to the LOT.</Paragraph>
    <Paragraph position="6"> Analysis is executed by principle of the best-first gr aph-sear'ching technique as follows= (i) Find the LOT which has the highest evaluation value.</Paragraph>
    <Paragraph position="7"> (2) Apply rewriting rules to the selected LOT.</Paragraph>
    <Paragraph position="8"> (3) If a rule is applicable, create new nodes (LOTs).</Paragraph>
    <Paragraph position="9"> (4) Assign the new evaluation values to the new LOTs by the above method. (The initial LOT value is the summation of the scores attached to words. ) (5) Go to (1).</Paragraph>
    <Paragraph position="10"> 2.3. Application of Rewriting Rules. The detail of the rule application sequence to a LOT which is selected by the best-first graph-searching technique is the following order= (1) From left elements of the LOT. {2) FrOm the rule which has the longest right side.</Paragraph>
    <Paragraph position="11"> (3) From the rule whose basic score is the largest.</Paragraph>
    <Paragraph position="12"> ( A B {I} ( X ~ Z } (2) ( X' Y' ) (3) ( x- ) (4) ( x Y z ) {5) ( X' Y' ) (6) ( X&amp;quot; )</Paragraph>
    <Paragraph position="14"> Fig. 6 An example of the application order of rewriting rules Fig. 6 shows a simple example of the rule application when rewriting rules have (x Y Z), (X Y ), and (X&amp;quot;) as their right side, and (X&amp;quot;), and the selected LOT is {A B C D). First (A B C) is matched with (X Y Z). If the matching is not successful, (A B) is matched with {X w yW). Tf the matching is not successful, {A) is matched with (X&amp;quot;). If the matching is not successful again, {B C D) is matched with (X Y Z), and so on.</Paragraph>
    <Paragraph position="15"> To speed up the rule applications, matching patterns which are right sides of rewriting rules are reconstructed in a tree structure such as shown in Fig.</Paragraph>
    <Paragraph position="16"> 7.</Paragraph>
    <Paragraph position="17"> original reconstructed rewriting rules rewriting rules</Paragraph>
    <Paragraph position="19"> In Fig. 7, if the first element of the LOT does not match with A, we do not need to test the rules rl - r3. So the rule r4 alone is tested for the application. By this reconstruction, the number of rules which are to be applied to a LOT is decreased qrately.</Paragraph>
    <Paragraph position="20"> 256 M. NAGAO and J. NAKAMURA</Paragraph>
    <Section position="1" start_page="253" end_page="253" type="sub_section">
      <SectionTitle>
2.4. Pruning Rule
</SectionTitle>
      <Paragraph position="0"> This parser is essentially a bottom-up parser, and there are cases that unnecessary expansions are executed. To minimize such unnecessery expansions, we introduced a mechanism of pruning such unnecessary nodes by certain pruning ~ules. For example, in the analysis of Japanese svntenc~ there must be ~ome verb phrase= (%~) to the right of a noun phrase (ME}, so %~e use the pruning rule shown in Fig. 8.</Paragraph>
      <Paragraph position="1"> It ~,atches with LOT, if LOT consists of sc~e trees, a tree N, NP or NP-C~ and trees which are not V, V-DA or VP in this order.</Paragraph>
      <Paragraph position="3"> (There must be V, ~.DA or VP in ~h~ ~igh~: of N, NP or NP-Co) ?igo 8 An example of the pruning ru!e The p~un!ng rules are described by matching patterns just the same as the right side of re~rit!ng rulesdeg They are matched with the whole LOT at the time that a LOT is created. If a pruning rule matches with the LOT, the node is pruned.</Paragraph>
    </Section>
    <Section position="2" start_page="253" end_page="253" type="sub_section">
      <SectionTitle>
2.5. Sententlal Pattern
</SectionTitle>
      <Paragraph position="0"> sententlal pattern (SP} expresses the global structure of a sentence.</Paragraph>
      <Paragraph position="1"> Fig. 9 shows examples of SPo  The top two lines are the LOTs which are intermediate structures from an input sentence:  grm~matlcal category name, not a tree structure. The elements of a sentential pattern are compared with the sequence of grammatical category names in a node. SP (1) ~p~esents that NP-Ck (JSEUPDTE-progr~,I-H~) is related to VP1 (the first embedded verb, Shuselsuru (mcdify)}. SP (2) represents that NP-Ck Is related to V-DA (main verb DA (is))o The ~a,:,~er assigns ~P-sco~s and SP-rule to a sentential Fattern. SP-score is a number such as shown in Fig. ~. ~his $~presses the plausibility of the styl~s of sentencesdeg in this exmnp).C: SP (i) is assigned the numerical vPS~ue: -I~ and SP (2) is essggned the value: +!~ as the SP-sco~:eo These t~o vaiue~ mean that~ when th~ main verb is V-~A, th~ first NP-C haPS tendency to be related to the main verb rather than to the first embedded verb. This SP-score is added to the evaluation value explained in section 2.2. Therefore~ analysis (1) takes precedence over anal-, sis (2) in hhis case.</Paragraph>
      <Paragraph position="3"> Fig. i0 An exmaple of SP-rule ~!'Igo i0 shows an example of SPruleo The sentential pattern whose SP-score is positive has at least one correct analysls. And a sequence of rule appllcatlon~ to the sententlal structure is guaranteed. S~-rules represent this sequence. However, it is not evident whether the sententlal pattern whose SP-score is negative has correct analyses, because it has at least one incorrect analysis. So we do not attach any SP-rule to it.</Paragraph>
      <Paragraph position="4"> SP-rule in Fig. I0 shows that we can get a correct analysis, if we apply rule-i - rule-4 to the LOT. Fig. 11 shows this process of rule applications. The sequential rule application of these four rules Is equivalent to a</Paragraph>
    </Section>
  </Section>
  <Section position="4" start_page="253" end_page="253" type="metho">
    <SectionTitle>
PARSER LEARNING ORDER OF REWRITING RULES 257
</SectionTitle>
    <Paragraph position="0"> rewriting rule shown in Fig. 12. But the rewriting rules Of the form shown in Fig. 10 are much better because the semantic check functions can be easily introduced to the simpler rules such as those in Fig. i0 rather than to such complex rules as those in Fig. 12.</Paragraph>
    <Paragraph position="1">  rewriting rule Each LOT is compared to sentential patterns from the first element of the LOT. The LOT is regarded as matched if the first part of the LOT matches a The parser changes the scores of sentential pattern, rewriting rules and SP-scores in the following way~</Paragraph>
  </Section>
  <Section position="5" start_page="253" end_page="253" type="metho">
    <SectionTitle>
3. Supervised Learning of Basic Scores, (i) Increase the scores of the rewrit-
</SectionTitle>
    <Paragraph position="0"> SP-scores and SP-rules ing rules and SP-scores on the path I from the root node to the success-To increase the efficiency of the ful node, and those on the pathes analysis, the parser controls basic which flow into the successful scores attached to rewriting rules, pathes.</Paragraph>
    <Paragraph position="1"> SP-scores and SP-rules. It is not easy (2) Decrease the scores of the rewritfor rule writers to assign scores to ing rules and SP-scores on the rewriting rules and to sentential first arcs of the pathes which flow patterns, and also to write SP-rules out the successful pathes.</Paragraph>
    <Paragraph position="2"> for a sentential pattern. We tried to SP-rules are gathered for each adjust these scores and to get SP-rules sentential pattern on the successful by the supervised learning in which the pathes by using the information in the user teaches the correctness of an a- search graph.</Paragraph>
    <Paragraph position="3"> nalysis to the parser.</Paragraph>
    <Paragraph position="4"> Fig. 13 shows an example of a 4. Result of Seme Experiments search graph when a sentence is analyzed. Each node of the search graph The sample sentences to be andcorresponds to a LOT. Each arc lyzed are taken fro~ a computer manual corresponds to a rule application. We in Japanese. About 150 sentences are can regard the LOTs on the path from used for the experiments. Conjunction the root node to the successful node as structures of noun phrases are useful structures, and the rewriting eliminated from these sentences. Among rules on the path as useful rules for the future analysis of similar sentential structure. On the other hand, other LOTs and rewriting rules in the search graph are regarded as useless to the future us~e; But ~e nodes and arcs \[i\] in Fig. 13 are not the direct reason of the failure. The direct cause for the failure comes from the nodes and ~cs \[ii\] in Pig. 13.</Paragraph>
    <Paragraph position="5">  150 sentences, 20 sentences are used for the supervised learning. These are selected ran&amp;:mly. The rewrfting rules are created from the gralmar proposed by Okutsu \[2\]. The number of rewriting rules is 54. The re~rlting rules in this experiment do not have the semantic check functions for s~pllclty.</Paragraph>
    <Paragraph position="6"> They are prepared to get the syntactic structures for a sentence.</Paragraph>
    <Paragraph position="7"> 4.1. Experiment I - Learning of Basic</Paragraph>
    <Section position="1" start_page="253" end_page="253" type="sub_section">
      <SectionTitle>
Scores of Rewriting Rules.
</SectionTitle>
      <Paragraph position="0"> To see the efficiency improvement of the analysis from the contribution oPS basic scores, SP-scOres and SP-rules are not used. The initial order of the rewriting rules is determined by random numbers. The initial basic scores are set the same value I for all rules. We adjusted basic scores 4 times, every time after 20 sentences for learning are analysed. We corpared the CPU-tlmes of the 2nd, 3rd and 4th analyses to the CPU-tlme of the let analysis. The result is shown in Table 3.</Paragraph>
      <Paragraph position="1">  Table 3 tells us that the basic scores of rewriting rule~ are not ~o useful for the improvement of the efficiency oPS analysis. The learned order oPS re-writing rules does not have a slgnlf~cant tendency. The reason Is that the structure of natural languages is recursive and the relative order of rules are more important to the anslysls than the over-all ordering, so that the basic scores cannot express the relative order.</Paragraph>
      <Paragraph position="2"> 4.2. Experiment 2 - The Effec~ of SP-sonres and SP-rules The learning of the SP~scores and SP-rules are done by enalys~ng the se~ of sample sentences once (20 Sentences selected amon 9 153 sentences r~n~ly}. Then the analysis of the set o~ 3~mpAe sentences (153 sentences) is d~e wit/~ and without using SP-soo~e~ ~ S~rules. The result of the experiment is in Table 4.</Paragraph>
      <Paragraph position="3"> Table 4 Effect of SP-scores and SP-rulee oYhe same SP not the same SP nu~Lber sentences~ 42 111 max.  |26.06~ 108.63% average \[ 19.23t 67.36% min.  |1.03% 9.46% (The values are the ratio of th~ analysis time with SP-scoree and S~ rules to the analysis time withoul them.) About 200 sententlal patterns are extracted frc, n the 20 sample sentences for learning. SP-zules are very useful for the sentences which have the same sententlal patterns, because the zewriting rules and their application sequence in the analysis of the sententlal pattern can be obtained from SP-rules which are defined from the past analysis, and no more trial search is necessary. 27.5% oPS sample sentences have the same ssntentlal patterns as the sentences foe learning. This n~ans that s(~e documents l~ke a computer manual contain very similar se~ences. Sententlal patterns and SP-rules are useful PSor the analysis oPS such documents. null</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>