A t3ottom-up Generation for Principle-based Grammars Using 
Constraint Propagation 
Masato Ishizaki 
NTT Communications and Information Processing Laboratories 
1-2356 Take Yokosuka-shi Kanagawa-ken, 238-03 Japan 
E-mail: islfizaki%nttnly.nt t.jp@relay.cs.net 
Abstract 
A bottom-up generation algorithm for 
principle-based grammars is proposed. 
Bottom-up generation has (1) an ineffi- 
ciency because of a high degree ofnonde- 
terminism, (2) a limitation caused by in- 
ability to process logical forms produced 
by grammar rules, and (3) an identity 
semantic problem. This paper describes 
a solution to these problems and imple- 
mentation issues for the algorithm using 
a constraint logic programming language. 
1 Introduction 
The generation of strings from logical forms was stud- 
ied intensively by \[Shi88,89\]\[Ca189\]. The study of 
this problem shed light not only on the efficiency and 
soundness of generation algorithms, but also on the 
descriptive appropriateness of grammar itself. 
A generation algorithm based ou the Early's 
method has proposed, which is capable of analyzing 
and generating sentences in a uniform architecture 
\[Shi88\]. In this architecture the criterion of"seman- 
tic monotonicity" is assumed to reduce fruitless gen- 
eration. As Shieber has mentioned, this method is 
still inefficient and limited, and ,instead, has shown 
an efficient and general top-down generation algo- 
rithm using the semantic-head concept 1 \[Shi89\]. 
An algorithm for Categorial Unification Gram- 
mar(C.UG) is mentioned in \[Ca189\]. As to the impor- 
tant ~:ole of lexicon, this grammar shares the same 
property as principle-based grammars, but difthrs in 
that CUG has grammar rules in the lexicon(This 
enables top-down generation with prediction as dis- 
cussed in \[Shi89\]). The identity semantic problem 
and the lexical indexing problem are some of the 
same problem noted in principle-based grammars. 
Generation based on Lexical Functional Gram- 
mar(LFG) is formalized in \[Wed88\]. He defined 
the derivability and the generability of f-structure- 
to-string and semantic-structure-to-string mapping. 
These concepts seem to be suitable for top-down 
lPrecisely his algorithm tMces a mixture of top-down and 
top-down way. 
strategies. His idea is intrigning in that discourse 
information, such as topic, can be reflected on an 
output string. Bottom-up generation also has the 
possibility of incorporating this information. This 
issue will be discussed later. 
I propose a bottom-up generation algorithm for 
principle-based grammars, which makes use of the 
same grammar as a parser. Bottom-up generation is 
inefficient due to a high degree of nondeterminism. 
It is limited by its inability to process se,nantic ex- 
pressions created by grammar rules. It also has an 
identity semantic problem. This paper describes the 
solution to these problems and the issues concerning 
implementation of the algorithm using a constraint 
logic programming language, cu-PROLOG. 
First, the problems of bottom-up generation are 
shown. Next the generation algorithm using con- 
straint propagation and solutions to these problems 
are considered. Third, some issues concerning imple- 
mentation and an example of this algorithm are men- 
tioned. Finally, the problem of logical form equiva- 
lence, lexical indexing, and controlled search using 
the discourse information are discussed. 
2 Problems of Principle-based 
Grammar Generation 
Principle-based grammars like tlead-driven Phrase 
Structure Grammar(HPSG) or Japanese Phrase 
Structure Grammar(JPSG) seem to be inadequate 
for top-down operations. The reasons for this are; 
(1) as principle-based grammars have few skeleton 
rules, top-down operations that are assumed to use 
rules are not adequate for them, 
(2) much information is in lexical items instead of 
grammar rules in principle-based gramm~Lrs, and 
(3) the construction of semantic expressions is intu- 
itively adequate for bottom-up operations ~. 
Therefore, an efficient bottom-up generator for 
principle-based grammars must be developed. 
Bottom-up generation has two advantages; it can 
use lexical information at an early phase, and can 
bottom-up strategy. But generation is fundamentMly driven in a 
2U:aification can be used for bl-dlrectional operations. However, if you need destructive semantic operations('destructive' means 
that a resultant semantic expressions cannot be constructed from base semantic expressions without these special functions), uni- 
ficatlon cannot be used for these operations. 
1 
188 
avoid the left recursion because of the property sim- 
ilar to bottom-up parsing, tIowever, bottom-up gen- 
eration has three problems\[Shi88,89\]\[Ca189\]; 
(1) inefficiency because of a high degree of nondeter- 
minism, 
(2) limitation which is caused by inability to process 
semantic expressions created by grammar rules, and 
(3) vestigial\[Shi88\], or identity semantic prob- 
lem\[Ca189\] (hereafter called identity semantic prob- 
lem). 
(2) and (3) relate to completeness and coherence 
problems\[Wed88\]. 
Inefficiency because of a high degree of nondeter- 
minism means that a naive bottom-up generation al- 
gorithm cannot use semantic information properly, 
and many semantic-irrelevant subexpressioas that 
are syntactically correct will result. 
One of the reasons that Shieber adopted a top- 
down strategy is that he insisted on the existence of 
logical tbrms produced by gramnmr rules. This as- 
sumpti0n means that sub-semantic expressions can- 
not be derived from resultant semantic ones that are 
different from normal ones. It is shown later that if 
we permit this kind of destructive semantic opera- 
tion, we cannot obtain an efficient algorithm. 
An identity semantic problem is common among 
generation algorithms using lexical-based grammars. 
The semantic expressions are classified into substan- 
tial eh'ments that contribute to the whole semantic 
expres.~ions, and functional elements(these are iden- 
tity semantic expressions) without influencing the 
whole ones. Examples of tile functional elements are 
complementisers in English, or case-marking postpo- 
sition ~n 3apanese. Simply speaking, the solution is 
to introduce these items at some time, but this also 
produces high inefficiency. 
3 Principle-based Grammar Gen- 
eration 
pendent of data, it does not incur such a problem. 
Hasida has shown that sentence analysis and syn- 
thesis can be described following a simple program 
with a constraint, constitttent 3 
struct( Category,X,Y); constituent(Category,X,Y). 
If active constraint solving techniques are applied 
to the problem, can this program be executed effi- 
ciently? Active constraint solving is equivalent to 
fold/unfold transformation \[Tud89\]. If the constraint 
clause is simply unfolded, then the number of clauses 
created will be the same as the number of lexical 
items. Presently this is not an efficient way 4. 
If passive constraint solving techniques are used, 
then how is data obtained? The answer is to predict 
the base lexical item, which is the core of a sentence, 
using a top-down prediction analogous to a bottom- 
up parsing technique. 
gen(cat(P,F,Aa,Au,Sc,Sem)) :- 
pred(cat(P,F,Aa,Au,Sc,Sem),Sem,BaseLex), 
gen1(Sem,BaseLex,NewBaseLex, 
cat(P,F,Aa,Au,Sc,Sem)). 
genl(Sem,BaseLex,NewBaseLex,Cat) 
verify(NewBaseLex,Cat). 
genl(Sem,BaseLex,NewBaseLex,Cat) 
getLex(Sem,BaseLex,BaseLexl), 
gen1(Sem,BaseLexl,NewBaseLex). 
getLex(Sem,BaseLex,NewBaseLex) '- 
getLexl(Sem,Lex); 
psr(Lex,BaseLex,NewBaseLex). 
getLex(Sem,BaseLex,NewBaseLex) "- 
getLexl(Sem,Lex), introduceFLex(FLex); 
psr(Lex,FLex,Lexl), 
psr(Lexl,BaseLex,NewBaseLex). 
Figure 1: The generation algorithm. 
3.1 Generation Algorithm Using Con- 
straint Propagation 
Natural language processing, such as sentence com- 
prehension and production, is thought of as con- 
straint satisfaction problems\[Has86\]. Constraint 
propagation techniques are very effective in these 
problems\[Din86\]. Constraint propagation techniques 
are classified into two methodologies: active con- 
straiut, which transforms constraints into more ef- 
ficient ones, and passive constraint, which is realized 
by the function such as freeze in Metalog. Passive 
constraint is similar to data-driven control, so if data 
does not arrive, the constraints are unsolved. As ac- 
tive constraint solving transforms constraints inde- 
This generation algorithm is sketched by the cu- 
PROLOG, or proiogIII notation ill figure 1. Tile 
predicate gen produces a sentence string from the 
term cat(P,F, Aa,Au, Sc, Sem,) 5, Tile term cat repre- 
sents a set of features: P is tile feature for part-of- 
speech, F is form such as verb inflection, Aa is ad- 
jacent node specification, Au is adjunct node specifi- 
cation, Sc is subcategorization information, and Sere 
is semantic information. The predicate prod antici- 
pates BaseLez that is the core of a sentence(normally 
head verb), using part-of-speech and semantic infor- 
mation. BaseLez has lexical and feature information. 
The predicate genl gets a lexical item, and applies 
principles to the item and the base item until produc- 
ing a sentence, getLez extracts a lexical item that is 
3This is rather misleading. An efficiency problem of generation and theoretical consideration about the problem are also 
mentioned in \[Has86 I. 
4If other techniques are developed, this consideration is not always correct. 
SAlthough this top predicate is a recognizer, constructing a tree or a string is easily realized with little effort. 
2 
189 
constrained by principles 6 using semantic informa- 
tion. introduceFLez extracts an identity semantic 
item that is constrained by principles. 
3.2 Counterarguments to Bottom-up 
Generation Deficiency 
3.2.1 InefRciency Due to High Degree of 
Nondeterminism 
C.onstraint propagation techniques in the previous 
section remedies nondeterminism of a bottom-up 
generation problem. An example of nondetermin- 
ism is the noun phrase generation in \[Shi89\]. If a 
NP occurs before a verb, different case NPs will be 
generated nondeterministically in figure 2. 
e 
Sp i verb 
e 
~Pn 
Figure 2: Nondeterminism in NP generation. 
~PZ 
e 
NPi 
e 
.~P. • J 
o 
e 
NP n 
psr (Daughter,ttead,Mother); 
member(Daughter,Head.subcategorization). 
psv represents a Mother --, Daughter Head rule. 
The predicate member is used as a constraint, which 
says the Daughter node is a member of a subcatego- 
rization of tire Head node. The constraint propaga- 
tion process is shown in figure 3. 
verb cor, st rair~s ,VPj 
due to its ~ubcate~orization l~t. 
~ verb 
Figure 3: Reduction of nondeterminism in NP 
generation. 
Constraints can also eliminate the irrelevant 
phonolotgical expressions shown in \[Ca189\], and com- 
pactly express tire order-variant subcategorization 
list in JPSG. 
3.2.2 :Limitation Owing to Special Semantic 
:Expressions 
Suppose a Mother --~ Daughter Head rule, where 
Mother.sent = f(Daughter.sem, Head.sem) 
and semantic function f destructively creates 
Mother.sem from Daughter.sere and Head.sere. That 
is, Daughter.sem and Head.sere cannot be predicted 
from Mother.sere. To get the value of Daughter.sere 
or Head.sere from Mother.sere, the inverse func- 
tion f-1 is needed. The implicit assumption of in- 
versability of the function\[Shi89\] is very severe, and a 
rather tricky feature structure must be constructed 
to escape the completeness problem. Therefore, it 
seems better to use another function in a semantic- 
monotonous framework instead of this one. 
However, we can easily modify the predicate pre- 
dict to get another head using the semantic informa- 
tion to which the function is applied, if this inverse 
function is obtained. 
3.2.3 An Identity Semantic Problem 
When using constraints to access fuuctlonal lexical 
items, an exhaustive search is not required. The in- 
sertion of Non-null constituents, such as case markers 
and fnnctional nouns, can be restricted using various 
constraints (syntactic, semantic information). For 
example, case markers that indicate the relationships 
between verbs and nouns are demanded by subcat- 
egori~ation information of the verbs. By using the 
constraint solving techniques, efficiency can be im- 
proved equal to or more than that of the top-down 
algorithm. Of Course since such occurrences(e.g, mdl 
constituents, or gap) cannot be restrained, this con- 
venient mechanism cannot be used. However, this 
situation is the same as top-down algorithms. 
4 Implementation 
4.1 Grammar FormaUsm and Imple- 
mentation Language 
Our algorithm exploits JPSG as phrase structure 
grammar formalism. The concept of JPSG inherits 
the fundamental property of HPSG. That is, JPSG 
makes use of a set of feature-value pairs, feature con- 
straints and unification to stipulate Japanese gram- 
mar instead of rewriting rules for terminal and non- 
terminal symbols. Subcategorization inforlnation is 
stored in lexical entries, instead of being stored in 
grammar rules. 
Logical forms are expressed by the semantic rep- 
resentation language proposed by \[Hob85\]. The dis- 
tinctive feature of this language is its simplicity of 
the form for discourse processing. The reason for 
this simple form is; (1)a conjunction of atomic predi- 
cates, (2)all variables are existentially quantified with 
the widest possible scope, and (3)there is no func- 
tions, functionals, nested quantifiers, disjunctions, 
negations, or modal or intensional operators. 
The algorithm is implemented using the cu- 
PROLOG developed in ICOT\[Tud89\]. The main fen- 
~A few phrase structure rules that observe principles, such as the head feature principle, are used in the program. 
3 
190 
~ure of this language is to adopt constraint unification 
instead of normal unification. This gives it more de- 
:;criptive power and more deelarativeness than nor- 
real prologs. The clause of cu-PROLOG is repre- 
sented as; 
h ,--- bl,b2,...,bn;el,c2,...,cn 
where h is a head, bl,b2,...,bn bodies, and cl,c2,_.,en 
constraints. 
p,~r(cat(P, F, \[\], \[\], Sc, Sere), 
cat(Pos, Form, \[cat(P, F, \[\], \[\], Sc, Sere)\], Au, SubCat, SEM), 
cat(Pos, Form, \[\], Au, SubCat, SEM),\[adjacent p\]). 
\]?sr(cat(P, F, \[\], Au, Sc, Sem), 
cat(Pos, Form, \[\], Adj, \[cat(P, F, \[\], /lu, So, Sem) lKest\], SEM), 
cat(Pes, Form, \[\], Adj, B.est, SEM),\[subcat p\] );sc cond(P,Pos). 
~,c cond(p.~v) . 
sc cond(adll,n). 
sc_cond(n~p) . 
diet (shorten,cat (n,n, \[\] , \[\] , \[\] , \[boy,X\] )). 
diet (tama,cat (n,n, \[\] , \[\] , \[\] , \[ball,X\])) . 
dict (ker, 
cat(v,vcr,\[\],\[\],S,\[\[kick,E,Sbj,Obj\],\[R1,Sbj\],\[R2,0bj\]\])); 
pp_wo ga(S,Obj,Sbj,R2,R1). 
d:i.et(ga,cat(p,ga,cat(n,F,lta,Au,Sc,Sem), \[\] , \[\] ,Sem)). 
d:tct(wo,cat(p,wo,cat(n,F,Aa,Au,Sc,Sem), \[\] , \[\] ,Sere)) . 
This grammar is shared with the parser mentioned in \[Tud89\] 
Figure 4: Part of grammar rules and a lexicon. 
shonen-ga tama-~0 kel'-u 
cat (v, .... \[\]. \[ \[kick, 1~:, X. Y\], \[boy, X\]. \[bait, ¥\]\]) 
cat (n ..... \[boy,X\]) cat (p, ga, .,~~~~~~~~ '~ "~ 
cat (v ...... \[cat (p, wo .... ), cat (p, ga .... ) \] .... ) 
Figure 5: The generation process example. 
This language has the ability to solve constraints 
in an active or passive way, but we use it for passive 
constraint solving. 
4.2 An Example 
Part of the grammar rules and a lexicon, and the 
process of generation from the logical form, 
\[ \[ kick,E,X,Y \] , \[ boy,X \] , \[ ball,Y \] \] 
are shown in figure 4 and figure 5. 
The flow of the algorithm is described as fol- 
lows; First, base lexical item (1)(a top down pre- 
diction) is predicted.This prediction instanciates tile 
packed subcategorization list, and constrains couno 
terpart conditions(2). Next, a lexical item is used 
to satisfy the constraints(2). Since this goal cannot 
be achieved by only the existing lexical item, a func- 
tional element is inserted that observes the restric- 
tion of the subcategoriz, ation list(3). As for (4) and 
(5), a similar process continues until a sentence is 
191 
produced. Finally we get an output 
shonen-ga tama-wo ker-u. 
a boy a ball kicks 
from the semantic expression. 
Figure 6 shows the result tree of real cu-PROLOG 
execution. 
v \[vcr\] : \[ \[kick ,E_2516 ,Y_2517 ,Y_2518\] , \[boy, Y_2517\] , \[ball ,Y_2518\] \] --- \[subcat_p\] 
--p \[ga\] : \[boy, Y_2517\] --- \[adj acenl;_p\] 
I 
I --n \[n\] : \[boy, Y_2517\] --- \[shonen\] 
I 
I __p \[ga, AJA{n \[n\] }\] : \[boy ,Y 2517\] --- \[ga\] 
__v\[vcr,SC{p\[wo\]}\] :\[\[kick,E_2516,Y_2517,Y_2518\],\[boy,Y_2517\],\[ball,Y 2518\]\] .... \[subcat p\] 
--p\[a\]:\[bal\]:,Y 2518\] --- \[adj acent p\] 
l 
I --n \[n\] : \[ball, Y_2518\] --- \[t ama\] 
I 
l__p\[wo,AJA{n\[n\]}\] :\[ball,Y_2518\]---\[wo\] 
_ V \[VCr, 
SC{p\[ga\],p\[wo\]}\]:\[\[kick,E 2516,Y_2517 ,Y 2518\],\[boy ,Y 2517\],\[ball,Y_2518\]\]---\[ker\] 
E = E_2516 X = Y_2517 Y : Y_2518 S = \[shonen,ga,tama,wo,keru\] 
Figure 6: The real execution result of the example using the cu-PROLOG. 
5 Remaining Problems 
5.1 The Problem of Logical Form 
Equivalence 
The problem of logical form equivalence has been dis- 
cussed in \[App87\]. This problem concerns genera- 
tion algorithms which are sensitive to logical forms. 
Namely, if an input semantic expression is converted 
by meaning postulates, different expressions with 
the same meaning are produced by different proce- 
dures.This problem occurs in the generation of ex- 
pressions that have quantificational ambiguities. 
A conversion using meaning postulates does not 
need syntactic or semantic information, but needs 
discourse information. Since generation strategies 
such as \[Shi89\], \[Ca189\] as well as ours use syntac- 
tic and semantic information, it is reasonable not to 
consider those operations. However, algorithms must 
have enough extendibility to reflect discourse infor- 
n'tation. 
5.2 Lexical Indexing Strategy 
Searching lexical items is very important for efficient 
algorithms. Metaknowledge about semantic expres- 
sions is necessary for this purpose, especially in the 
case of complex ones\[Cal89\]. 
This problem is not peculiar to generation. For 
example, discourse processing in which various infer- 
ences are executed by using the semantic expressions 
has the same problem. To cope with this problem, 
\[Hob85\] has proposed more simple logical forms. I 
also consider this a good idea for generation. 
5.3 Controlling Search Using Dis- 
course Information 
Many sentences corresponding to one meaning can 
be generated by our algorithm(or other algorithms). 
Idealistically all sentences are distinctively produced 
according to other information such ~s discourse in- 
formation. Our algorithm has the possibility for eas- 
ily realizing this mechanism. 
Suppose that an information uni~ agrees with a 
predicate in the semantic information. Control of the 
element position is realized by solving the constraint 
of the older information in turn r. Passivisation in 
Japanese is achieved by controlling the insertion of 
functional elements. 
Transformational Grammar(the antecedent of 
Parameter and Principle theory) indicates the inter- 
esting piece of data that have many sentences with 
the same meaning. This concerns the position of the 
elements and the introduction of functional elements. 
As mentioned above, our algorithm is capable of re- 
fleeting discourse information on surface structures 
because of constraints. 
~This assumption is problematic because correspondence is not guaranteed and if r~ semantic element is uniquely mapped to a 
lexical entry, the position is directly designated by the order of the semantic elements. 
5 
192 
The algorithm proposed in \[Wed88\] can generate 
sentences that reflect discourse phenomena such as 
topicalization in LFG, but the formalization of topic 
greatly helps to simplify the algorithm. 
6 Conclusion 
In this paper an efficient bottom-up generation algo- 
rithm for principle-based grammars using constraint 
propagation is proposed, and a solution to bottom- 
up generation problems is mentioned. Issues about 
implementation and an example processed by the al- 
gorithm are also shown. Both the parser\[Tud89\] and 
the generator use the same grammar, that is, the 
grammar is reversible. 
Since problems not inherent in bottom-up gener- 
ation are connected to the logical form problem, or 
the knowledge representation problem, they should 
be discussed more deliberately from the viewpoint of 
generation. 
Acknowledgement 
The author wishes to extend his sincere gratitude to 
Yoshihiko Hayashi, Tsuneaki Kato, and Gen-ichiro 
Kikui for their comments on the initial idea of this 
generation algorithm. The author also wishes to ex- 
press his indebtness to Tuda ttirosi, and Hasida Koiti 
of ICOT for permitting him to use the cu-PROLOG. 
Thanks are also due to Dr. Terashima, Mr. Sakama, 
Mr. tfigashida, Mr. Shimazaki and all the members 
of the Natural Language Processing Division for their 
cooperation. 

References 

lApp87\] Douglas E. Appelt. 1987. "Bidirectional 
grammars and the design of natural language gen- 
eration systems", Theoretical Issues in Natural 
Language Processing - 3, New Mexico, 185-191. 

\[Cal89\] Jonathan Calder, Mike Reape, and Henk 
Zeevat. 1989. "An Algorithm for Generation in 
Unification Categorial Grammar", Proceedings of 
the 4tlh Conference of the European Chapter of 
the Association for Computational Linguistics, 
Manchester, 10-12 April, 233-240. 

\[Din86\] Dincbas, M. 1986. "Constraints, Logic Pro- 
gramming and Deductive Databases", Proceed- 
ings of France-Japan Artificial Intelligence and 
Computer Science Symposium 86, Tokyo, 30 
November-3 December, 1-27. 

\[Has87\] Hasida Koichi and Isizaki Shun. 1987. "De- 
pendency Propagation: a Unified Theory of Sen- 
tence Comprehension and Generation", Proceed- 
ings of the 11th International Conference on 
Computational Linguistics, Milan, 23-28 August, 
664-670. 

\[Hob85\] Itobbs, R. 1985. "Ontological Promiscuity", 
23rd Annual Meeting of the Association for Com- 
putational Linguistics, New Jersey, 61-69. 

\[Po187\] Carl Pollard and Ivan A.Sag. 1987. "An 
Information-based Syntax And Semantics", Vol 1 
Fundamentals, CSLI Lecture Notes Number 13. 

\[Gun87\] Takao Gunji. 1987. "Japanese Phrase Struc- 
ture Grammar", D.Reidel Publishing Company. 

\[Shi88\] Stuart M. Shieber. 1988. "A Uniform Archi- 
tecture for Parsing and Generation", Proceedings 
of the 12th International C.onference on C, om- 
putational Linguistics, Budapest, 22-27 August, 
614-619. 

\[Shi89\] Stuart M. Shieber, Gertjan van Noord, 
Robert C. Moore, and Feraando C.N.Pereira. 
1989. "A Semantic-Head-Driven Generation Al- 
gorithm for Unificati0n-Based Foriualisms", 27th 
Annual Meeting of the Association for Computa- 
tional Linguistics, British Columbia, 26-29 June, 
7-17. 

\[Tud89\] Tuda Hirosi, Hasida Koiti, and Sirai 
Hidetosi. 1989. "JPSG Parser on Constraint 
Logic Programming", Proceedings of the 4th 
Conference of the European chapter of tile As- 
sociation for Computational Linguistics, Manch- 
ester, 10-12 April, 95-102. 

\[Wed88\] Jurgen Wedekind. 1988. "Generation as 
Structure Driven Derivation", Proceedings of the 
12th International Conference on Computational 
Linguistics, Budapest, 22-27 August, 732-737. 
