COLING 82, J. Horeck) (¢d,) 
North-Holland Publishing Company 
© Academia, 1982 
A Parser Which Learns the Application Order 
of Rewriting Rules 
Makoto Nagao, Jun-ichi Nakamura 
Department of Electrical Engineering 
Kyoto University 
Sakyo, Kyoto 
Japan 
0. Introduction 
The efficiency of syntactic analy- 
sis by using a set of rewriting rules 
is greatly influenced by the order or 
the arrangement of the rules for the 
application. There are some trials 
which subdivide the set of rules into 
subsets and specify the sequence of 
rule applications, thus avoiding the 
useless rule applicagions \[i\]. But the 
subdivision of the rule set and the 
specification of the sequence of rule 
applications are not so easy for the 
establishment of the most efficient a- 
nalysis system. 
We have developed a rewriting rule 
system which can manipulate arbitrary 
list of trees. The control mechanism of 
this system can adjust the weight of 
the rewriting rules, and can analyze 
the most plausible sentential structure 
f t, thus realizing the fast syntac- 
t analysis. The system learns (so to 
s\[ ) the weight of importance of the 
reutiting rules during the analysis of~ 
input sentences. 
I. Objectives of the Parser 
We designed a new syntactic analy- 
sis system (a parser) with the follow- 
ing objectives. 
(i) The function of rewriting rules 
must be powerful enough to handle a 
list of trees and to express trans- 
formational rules, 
(2) All the possible sentential struc- 
tures must be obtained for an input 
sentence in the sequence that the 
most plausible one is analyzed 
first. 
(3) The analysis must be efficient e- 
nough for practical applications. 
(4) The syntactic parser must have a 
learning mechanism as to the ap- 
plication sequences of rewriting 
rules to obtain the efficiency of 
analysis. 
2. Method of Analysis 
The input data for this parser is 
assumed as a word sequence which is the 
output of a morphological analysis. The 
output from this parser is a tree 
structure. The analysis is controlled 
by the best-first graph-searching 
technique about the rule applications. 
2.1. Description of Rewriting Rules 
The rewriting rules transform a 
list of trees into a list of trees. An 
example of the rewriting rule in this 
parser is shown in Fig. 1. It shows 
that if there is a symbol sequence com- 
posed of a tree not-V(erb), a tree 
N(oun) P(hrase), a tree 
C(ase-particle), and a tree 
not-A(dverbial)-P(article) in this 
order, this is transformed into a tree 
NP-C. 
NP-C Y 
/~ ~--- X NP C Y 
NP C (not V) (not AP) 
NP: Noun Phrase C:Case-particle 
V: Verb AP: Adverbial- 
Particle 
Fig. 1 An example of the rewriting 
rule 
The right side of rewriting rules 
is a matching pattern which is to be 
found in the given input symbol string. 
Table 1 shows the function symbols to 
describe the matching patterns. By 
using these function symbols, it is 
possible to specify the repetition of 
pattern elements, to assign data to a 
variable, and so on. Tt is also 
253 
254 M. NAGAO and .L NAKAMURA 
possible to check the input data by 
using user-defined functions. These 
functions enable us to describe complex 
syntactic functions, semantic rela- 
tions, and many heuristic checks. 
From ? 
# 
(%F fn al . .. an 
(%# at xl .,. xn 
Table I Function symbols of the 
matching patterns 
I Function Match an arbitrary tree• 
Hatch any number of arbitrary trees. 
an) Evaluate function= fn whose arguments are 
a Corresponding tree, al, 
•.., and an. When the Value is not NIL, matching 
succeed. 
xn) Match any number o4 lists of matching patterns xl 
• .. xn. Trees are assigned to variable at. 
(%A xl ... xn) Matching succeeds if all xl, ..., 
xn are matched to a tree. 
(%O xl ... xn\] Matching succeeds if one 
(%N x) 
of xl, .•., xn is matched to a tree. 
Matching succeeds if x is not matched to a tree. 
Table 2 Function symbols of the 
creation patterns 
Form Function at If at is a variable, then 
its value, otherwise at itsel~o 
(*F fn xl .. xn) The value Of the function: fn whose arguments are xl, 
• --t xn. (*S at x) The value of a generation 
Of x assigned to the variable 
at° 
The left side of rewriting rules 
is a creation pattern of new syntactic 
structures. Table 2 shows the function 
symbols for structure creation. User- 
defined functions can also be used to 
check certain relations in this crea- 
tion pattern. We can generate an 
arbitrary tree structure by this re- 
writing rule system. 
NP-Ct NP.-CJ NP-Ck VF 
.F-ci ~ --~c sP-ci NF-CJ NF-Ck VE 
NP-C~ NP-Ck VP NP-C: Noun Phrase + Case-partlcle 
NP-Ct MP-C~ ~} VP: Verb Phrase S: Sentence 
NP-Ck VP 
Fig. 2 An example of generating 
three left sides from a right side 
As shown in Fig. 2, we can specify 
arbitrary numbers of structures in the 
left side for the same right side in a 
rewriting rule. 
Each rewriting rule has a weight 
{basic score) and a function 
(fittedness function). The basic score 
is a static weighting measure which ~e- 
flects the importance of a rule co~- 
pared to the other ruleG of the same 
category. The basic score is adjusted 
by a learning process which will be ex- 
plained in section 3. The 'fittedness 
function gives a dynamic weighting 
measure which expresses the fittedness 
of the rule application to a sentential 
structure. The function is a user- 
defined one which can use the data in 
both the right side and the left sides 
of the rewriting rules. 
The basic score and the fittedness 
function are used for the sequence con- 
trol of rule applications in the best- 
first graph-searching, which is the es- 
sential strategy to get the most 
plausible structural analysis first. 
2.2. Flow of Analysis 
Fig. 3 shows an intermediate 
structure in the cOurse of a sentence 
analysis. 
( NP-C 
Datiset 
vP TENSE ) I 
V 
Sh te sur U 
(specify) 
Fig. 3 The structure of a sentence 
during the analysis {LOT) 
This structure is represented by a 
list of trees. We call this structure 
as a LOT {List of Trees). An analysis 
step is an application of a rewriting 
rule to a LOT as shown in Fig. 4. which 
changes the content of the LOT. 
( ~ c 
{ NP C I 
N 
( NF-C 
NF C I 
N 
v TENSE ) 
NP <--- N 
V TENSE ) 
~N~-C <--- NF C 
V TENSE } 
Fig. 4 Progress of an analysis 
PARSER LEARNING ORDER OF REWRITING RULES 255 
To obtain the result of an analy- 
sis one by one in the order of 
plausibility, we use the best-first 
graph-searchlng technique. If we regard 
a LOT as a node in a search graph, the 
new LOT created by the application of a 
rewriting rule to an old LOT is a sis- 
tee node. When several rules are ap- 
plicable to a LOT or the rule has 
several left sides, the same number of 
sister nodes are created from one moth- 
er node. The progress of analysis can 
be represented by an expansion tree (in 
general, by a graph) as shown in Fig. 
5. 
I 
(A ZI /%) LOT - node 
/ ~ ~- ... Application of 
( A /k ) (Zi A) rewriting rule 
/ ~ \ ... / "%'~... = expans£on 
(~) 
Fig. 5 Search tree 
This tree can be regarded as a 
search tree. We expand the node which 
has the highest evaluation value (the 
score assigned to the LOT} first. The 
expansion is the application of a re- 
writing rule to a LOT. The evaluation 
value is obtained by the summation of 
the following four values: 
(1) the evaluation value of the mother 
node. 
(2 the basic score which is attached 
to the applied rule. 
(3) he value obtained from the 
t ittedness function which ~ is 
attached to the applied rule. 
(4) the score of the sentential pattern 
(SP. which will be explained in 
section 2.5), if it matches to the 
LOT. 
Analysis is executed by principle 
of the best-first gr aph-sear'ching 
technique as follows= 
(i) Find the LOT which has the highest 
evaluation value. 
(2) Apply rewriting rules to the se- 
lected LOT. 
(3) If a rule is applicable, create new 
nodes (LOTs). 
(4) Assign the new evaluation values to 
the new LOTs by the above method. 
(The initial LOT value is the 
summation of the scores attached to 
words. ) 
(5) Go to (1). 
2.3. Application of Rewriting Rules. 
The detail of the rule application 
sequence to a LOT which is selected by 
the best-first graph-searching 
technique is the following order= 
(1) From left elements of the LOT. 
{2) FrOm the rule which has the longest 
right side. 
(3) From the rule whose basic score is 
the largest. 
( A B {I} ( X ~ Z } 
(2) ( X' Y' ) 
(3) ( x- ) (4) ( x Y z ) 
{5) ( X' Y' ) (6) ( X" ) 
(7) ( X' Y' ) (8) ( X" ) 
(9) ( x- ) 
C D ) <---LOT 
application 
order of 
rewriting 
rules 
Fig. 6 An example of the application 
order of rewriting rules 
Fig. 6 shows a simple example of 
the rule application when rewriting 
rules have (x Y Z), (X Y ), and (X") 
as their right side, and (X"), and the 
selected LOT is {A B C D). First (A B 
C) is matched with (X Y Z). If the 
matching is not successful, (A B) is 
matched with {X w yW). Tf the matching 
is not successful, {A) is matched with 
(X"). If the matching is not successful 
again, {B C D) is matched with (X Y Z), 
and so on. 
To speed up the rule applications, 
matching patterns which are right sides 
of rewriting rules are reconstructed in 
a tree structure such as shown in Fig. 
7. 
original reconstructed 
rewriting rules rewriting rules 
rl ( A B C ) A->B ->C 
r2 (A B D)--> I ~ D 
r3 (A E) 
r4 ( F G ) F ->G 
Fig. 7 Reconstruction of rewriting 
rules 
In Fig. 7, if the first element of the 
LOT does not match with A, we do not 
need to test the rules rl - r3. So the 
rule r4 alone is tested for the ap- 
plication. By this reconstruction, the 
number of rules which are to be applied 
to a LOT is decreased qrately. 
256 M. NAGAO and J. NAKAMURA 
2.4. Pruning Rule 
This parser is essentially a 
bottom-up parser, and there are cases 
that unnecessary expansions are ex- 
ecuted. To minimize such unnecessery 
expansions, we introduced a mechanism 
of pruning such unnecessary nodes by 
certain pruning ~ules. For example, in 
the analysis of Japanese svntenc~ 
there must be ~ome verb phrase= (%~) to 
the right of a noun phrase (ME}, so %~e 
use the pruning rule shown in Fig. 8. 
It ~,atches with LOT, if LOT consists of 
sc~e trees, a tree N, NP or NP-C~ and 
trees which are not V, V-DA or VP in 
this order. 
(# (%0 (? N #) (? NP #) (? N~-C ~)) 
(%# NIL (%N (%0 (? V #) 
(? V-DA #) 
(? ~ #)))) ) 
(There must be V, ~.DA or VP in ~h~ 
~igh~: of N, NP or NP-Co) 
?igo 8 An example of the pruning ru!e 
The p~un!ng rules are described by 
matching patterns just the same as the 
right side of re~rit!ng rules° They are 
matched with the whole LOT at the time 
that a LOT is created. If a pruning 
rule matches with the LOT, the node is 
pruned. 
2.5. Sententlal Pattern 
sententlal pattern (SP} expresses 
the global structure of a sentence. 
Fig. 9 shows examples of SPo 
(I) ( S-OBJi NP V-DA TENSE ) : -i 
(2) ( NP-Ck S-OBJj NP V-DA TENSE ) : +1 
(a) Sentential patterns 
(i) ( S-OBJi MP V-DA TENSE ) 
NP-Ck NP-Cm VEI TENSE 
(2) ( NP-Ck S-OB ._Jj NP V-DA TENSE ) 
NP-Cm VPI TENSE 
(b) Corresponding LOTs 
( NE-Ck T~P-Cm VPI TENSE V.-DA "TENSE ) 
(c) Original LOT 
Fig. 9 Examples of Sententlal PatternE 
(sp) 
The top two lines are the LOTs 
which are intermediate structures from 
an input sentence: 
(NP-Ck NP-Cm VPI T~SE NP V-DA TENSE) 
JSEUPDTE-pEogram Ha Source- 
program-llbrary Wo Shuselsuru(modify) 
Dataset-utillty Dears(Is). 
(JSEUPDTE program is a Dataset 
utility which modifies source program 
libraries.) 
Each element of sentential pattern is a 
grm~matlcal category name, not a tree 
structure. The elements of a sentential 
pattern are compared with the sequence 
of grammatical category names in a 
node. SP (1) ~p~esents that NP-Ck 
(JSEUPDTE-progr~,I-H~) is related to VP1 
(the first embedded verb, Shuselsuru 
(mcdify)}. SP (2) represents that NP-Ck 
Is related to V-DA (main verb DA (is))o 
The ~a,:,~er assigns ~P-sco~s and 
SP-rule to a sentential Fattern. SP- 
score is a number such as shown in Fig. 
~. ~his $~presses the plausibility of 
the styl~s of sentences° in this exmn- 
p).C: SP (i) is assigned the numerical 
v£~ue: -I~ and SP (2) is essggned the 
value: +!~ as the SP-sco~:eo These t~o 
vaiue~ mean that~ when th~ main verb is 
V-~A, th~ first NP-C ha£ tendency to 
be related to the main verb rather than 
to the first embedded verb. This SP- 
score is added to the evaluation value 
explained in section 2.2. Therefore~ a- 
nalysis (1) takes precedence over anal-, 
sis (2) in hhis case. 
( NP-C ~ C VP TENSE ) : SP 
( ( rule-i 2 ) 
' rule--2 1 ) : SP-rule 
( rule-3 i ) 
( ru!e-4 ! ) ) 
Fig. i0 An exmaple of SP-rule 
~!'Igo i0 shows an example of SP- 
ruleo The sentential pattern whose SP- 
score is positive has at least one 
correct analysls. And a sequence of 
rule appllcatlon~ to the sententlal 
structure is guaranteed. S~-rules rep- 
resent this sequence. However, it is 
not evident whether the sententlal 
pattern whose SP-score is negative has 
correct analyses, because it has at 
least one incorrect analysis. So we do 
not attach any SP-rule to it. 
SP-rule in Fig. I0 shows that we 
can get a correct analysis, if we apply 
rule-i - rule-4 to the LOT. Fig. 11 
shows this process of rule applica- 
tions. The sequential rule application 
of these four rules Is equivalent to a 
PARSER LEARNING ORDER OF REWRITING RULES 257 
rewriting rule shown in Fig. 12. But 
the rewriting rules Of the form shown 
in Fig. 10 are much better because the 
semantic check functions can be easily 
introduced to the simpler rules such as 
those in Fig. i0 rather than to such 
complex rules as those in Fig. 12. 
NP-C NP C VP TENSE ) I .... NP-C <--- NP C ( rule-I ) 
NP-C NP-C VP TENSE } | .... S-NUCL <--- NP-C NP-C VP ( rule-2 
S-NUCL TENSE ) | .... S-OBJ <--- S-NUCL TENSE ( rule-3 } 
S-OBJ ) .... S <--- S-OBJ ( rule-4 ) 
S } 
Pig. 11 An example of the SP-rule 
application 
S i 
S-OBJ 
<- NP-C NP C VP TENSE 
S-NUCL TENSE 
~ NI~.C VP A 
NP C 
Fig. 12 An example of the equivalent 
rewriting rule 
Each LOT is compared to sentential 
patterns from the first element of the 
LOT. The LOT is regarded as matched if 
the first part of the LOT matches a The parser changes the scores of 
sentential pattern, rewriting rules and SP-scores in the 
following way~ 
3. Supervised Learning of Basic Scores, (i) Increase the scores of the rewrit- 
SP-scores and SP-rules ing rules and SP-scores on the path 
I from the root node to the success- 
To increase the efficiency of the ful node, and those on the pathes 
analysis, the parser controls basic which flow into the successful 
scores attached to rewriting rules, pathes. 
SP-scores and SP-rules. It is not easy (2) Decrease the scores of the rewrit- 
for rule writers to assign scores to ing rules and SP-scores on the 
rewriting rules and to sentential first arcs of the pathes which flow 
patterns, and also to write SP-rules out the successful pathes. 
for a sentential pattern. We tried to SP-rules are gathered for each 
adjust these scores and to get SP-rules sentential pattern on the successful 
by the supervised learning in which the pathes by using the information in the 
user teaches the correctness of an a- search graph. 
nalysis to the parser. 
Fig. 13 shows an example of a 4. Result of Seme Experiments 
search graph when a sentence is ana- 
lyzed. Each node of the search graph The sample sentences to be and- 
corresponds to a LOT. Each arc lyzed are taken fro~ a computer manual 
corresponds to a rule application. We in Japanese. About 150 sentences are 
can regard the LOTs on the path from used for the experiments. Conjunction 
the root node to the successful node as structures of noun phrases are 
useful structures, and the rewriting eliminated from these sentences. Among 
rules on the path as useful rules for 
the future analysis of similar senten- 
tial structure. On the other hand, oth- 
er LOTs and rewriting rules in the 
search graph are regarded as useless to 
the future us~e; But ~e nodes and 
arcs \[i\] in Fig. 13 are not the direct 
reason of the failure. The direct cause 
for the failure comes from the nodes 
and ~cs \[ii\] in Pig. 13. 
act e~d~s1/ / ii/ 
.-'~--: / ./~ 
failed ~t ~d~ s~ failed 
~z ~iting Rule 
SP: Sententlal Pattern 
Fig. 13 Relation ~t~en the state 
of the expa~ion and 
failure or success of the analysis 
258 M. NAGAO and J. NAKAM~ltA 
150 sentences, 20 sentences are used 
for the supervised learning. These are 
selected ran&:mly. The rewrfting rules 
are created from the gralmar proposed 
by Okutsu \[2\]. The number of rewriting 
rules is 54. The re~rlting rules in 
this experiment do not have the seman- 
tic check functions for s~pllclty. 
They are prepared to get the syntactic 
structures for a sentence. 
4.1. Experiment I - Learning of Basic 
Scores of Rewriting Rules. 
To see the efficiency improvement 
of the analysis from the contribution 
o£ basic scores, SP-scOres and SP-rules 
are not used. The initial order of the 
rewriting rules is determined by random 
numbers. The initial basic scores are 
set the same value I for all rules. We 
adjusted basic scores 4 times, every 
time after 20 sentences for learning 
are analysed. We corpared the CPU-tlmes 
of the 2nd, 3rd and 4th analyses to the 
CPU-tlme of the let analysis. The re- 
sult is shown in Table 3. 
Table 3 Effect of basic scores 
12nd/let 3rd/Ist 4th/Ist 
max.| 99.37~ 102.10% 108.78% 
averaq~ 94.62% 96,75~ 96.47% 
mln.| 87.69% 87°88% 89.49% 
(The values are the ratio o£ th( 
2PO-time per word.) 
Table 3 tells us that the basic scores 
of rewriting rule~ are not ~o useful 
for the improvement of the efficiency 
o£ analysis. The learned order o£ re- 
writing rules does not have a slgnlf~- 
cant tendency. The reason Is that the 
structure of natural languages is re- 
cursive and the relative order of rules 
are more important to the anslysls than 
the over-all ordering, so that the ba- 
sic scores cannot express the relative 
order. 
4.2. Experiment 2 - The Effec~ of 
SP-sonres and SP-rules 
The learning of the SP~scores and 
SP-rules are done by enalys~ng the se~ 
of sample sentences once (20 Sentences 
selected amon 9 153 sentences r~n~ly}. 
Then the analysis of the set o~ 3~mpAe 
sentences (153 sentences) is d~e wit/~ 
and without using SP-soo~e~ ~ S~- 
rules. The result of the experiment is 
in Table 4. 
Table 4 Effect of SP-scores and 
SP-rulee 
oYhe same SP not the same SP 
nu~Lber 
sentences~ 42 111 
max. | 26.06~ 108.63% 
average \[ 19.23t 67.36% 
min. | 1.03% 9.46% 
(The values are the ratio of th~ 
analysis time with SP-scoree and S~ 
rules to the analysis time withoul 
them.) 
About 200 sententlal patterns are 
extracted frc, n the 20 sample sentences 
for learning. SP-zules are very useful 
for the sentences which have the same 
sententlal patterns, because the ze- 
writing rules and their application 
sequence in the analysis of the senten- 
tlal pattern can be obtained from SP- 
rules which are defined from the past 
analysis, and no more trial search is 
necessary. 27.5% o£ sample sentences 
have the same ssntentlal patterns as 
the sentences foe learning. This n~ans 
that s(~e documents l~ke a computer 
manual contain very similar se~ences. 
Sententlal patterns and SP-rules are 
useful £or the analysis o£ such docu- 
ments. 
5. Conclusion 
The experiments to examine the 
effect of lea~nlng are performed. The 
results of ~he experiment shows that 
SP-rules a~ very useful. Th~s ~eans 
that ~hls p~Eser can learn the s~yle of 
the sentences an~ can increase the ef- 
£1c~ncy of &nalyels when the senten- 
tlal structures o£ the texts in the 
partlcular field are ~estricted. 
This parser is implemente~ ~ LZSP 
on ~ACOM M-2O0 in Com~uter Cen~eE o£ 
Kyoto University. 

References

\[I\] Boltet, C., Aut~tlc ~rc~uct~n of 
CF an~ CS-a~ly~ using. A General 
sclen~Iflque ~u ~I~ de 
Gr~;~eble, I%79o 

\[2~ Okut~u, ~. ~ Sei~el ~Ipp~o ~un~o- 
