CATEGORIAL AND NON-CATEGORIAL LANGUAGES 
Joyce Friedman 
Ramarathnam Venkatesan 
ABSTRACT 
Computer Science Department 
Boston University 
111 Cummington Street 
Boston, Massachusetts 02215 USA 
PREL1MIN A R IES 
We study the formal and linguistic proper- 
ties of a class of parenthesis-free categorial 
grammars derived from those of Ades and Steed- 
man by varying the set of reduction rules. We 
characterize the reduction rules capable of gen- 
erating context-sensitive languages as those having 
a partial combination rule and a combination rule 
in the reverse direction. We show that any 
categorial language is a permutation of some 
context-free language, thus inheriting properties 
dependent on symbol counting only. We compare 
some of their properties with other contem- 
porary formalisms. 
INTRODUCTION 
Categorial grammars have recently been the topic 
of renewed interest, stemming in part from their use as 
the underlying formalism in Montague grammar. While 
the original categorial grammars were early shown to be 
equivalent to context-free grammars, 1, 2, 3 modifications 
to the formalism have led to systems both more and less 
powerful than context-free grammars. 
Motivated by linguistic considerations, Ades and 
Steedman 4 introduced categorial grammars with some 
additional cancellation rules. Full cancellation rules 
correspond to application of functions to arguments. 
Their partial cancellation rules correspond to functional 
composition. The new backward combination rule is 
motivated by the need to treat preposed elements. They 
also modified the formalism by making category symbols 
parenthesis-free, treating them in general as governed 
by a convention of association to the left, but violat- 
ing this convention in certain of the rules. 
This treatment of categorial grammar suggests a 
family of eategorial systems, differing in the set of can- 
cellation rules that are allowed. Earlier, we began a 
study of the mathematical properties of that family of 
systems, s showing that some members are fully 
equivalent to context-free grammars, while others yield 
only a subset of the context-free languages, or a super- 
set of them. 
In this paper we continue with these investigations. 
We characterize the rule systems that can obtain 
context-sensitive languages, and compare the sets of 
categorial \]ar~guages with the context-free languages. 
Finally, we discuss the linguistic relevance of these 
results, and compare categorial grammars with TAG 
systems i, this regard. 
A categorial grammar under a set R of reduction 
rules is a quadruple CGR (VT, VA, S, F), whose ele- 
ments are defined as follows: VT is a finite set of mor- 
phemes. VA is a finite set of atomic category symbols. 
S EVA is a distinguished element of VA. To define F, 
we must first define CA, the set of category symbols. 
CA is given by:i) ifAEVA,thenA ECA;ii) ifX EUA 
and A EVA, then X/A ECA; andiii) nothing elselsin 
CA . F is the lexicon, a function from VT to 2 ea such 
that for every aEVT, F(a) is finite. We often write 
CGR to denote a categorial grammar with rule set R, 
when the elements of the quadruple are known. 
Notation: Morphemes are denoted by a, b; mor- 
pheme strings by u,v,w. The symbols S,A,B,C 
denote atomic category symbols, and U. V, X, Y 
denote arbitrary (complex) category symbols. Complex 
category symbols whose left-most symbol is S (symbols 
"headed" by S) are denoted by Xs, Ys. Strings of 
category symbols are denoted by z, y. 
The language of a categorial grammar is determined 
in part by the set R of reduction rules. This set can 
include any subset of the following five rules. In each 
statement, A EVA, and 
U/A,A/U,A/V, VIA E CA. 
(1) (F Rule) The string of category symbols U/A A 
can be replaced by U. We write: U/A A---*U; 
(2) (FP Rule) The string U/A A/V can be 
replaced by U /V. Wewrite: U /A A/V-*U/V; 
(3) (B Rule) The string A V/A can be replaced 
by U. We write:A U/A~U; 
(4) (Bs Rule) Same as B rule, except that U is 
headed by S. 
(5) (BP Rule) The string A/U V/A can be 
replaced by V/U. We write: A/U V/A--*V/U. 
If XY ---,Z by the F-rule , XY is called an F-redex. 
Similarly, for the other four rules. Any one of them may 
simply be called a redex. 
The reduction relation determined by a subset of 
these rules is denoted by => and defined by: if X Y --* Z 
by one of the rules of R, then for any a, /~ in CA* , 
aXY/3 >aZ/3. The reflexive and transitive closure of 
the relation -> is =>*. A morpheme string 
w=wlu,~" "'w, is accepted by CGR(VT, VA,S,F) 
if there is a category string z = X1X2 "" • X, such that 
XiEF(w,) for each i=l,2,'--n, and x =>* S. The 
language L(CGR) accepted by CGR(VT, VA,S,F) 
is the set of all morpheme strings that are accepted. 
75 
I. NON-CONTEXT-FREE CATEGORIAL 
LANGUAGES 
In this section we present a characterization 
theorem for the categorial systems that generate only 
context-free languages. 
First, we introduce a lexicon FEQ that we will show 
has the property that for any choice R of metarules any 
string in L(CGR) has equal numbers of a,b, and c. 
We define the lexicon FEQ as FEQ (a ) = {A }, 
FEQ(b) = {BI, F~Q(c) ={C/A/C/B, C/D}, 
FEQ (d ) {D}, FEQ(e)={S/A/C/B}. 
We will also make use of two languages on the 
alphabet {a,b,e,d, e} Ll={a"db "e c ~ In >/1 },and 
LEQ = {w ! #a = #b = #c >1 1,#d =#e = 1}. 
A lemma shows that with any set R of rules the lex- 
icon FEQ yields a subset of LEQ. 
Lemma 1 Let G be -any categorial grammar, 
CGR(VT,VA,S,FEQ), where VT ={a,b,c,d,e}, 
VA = {S,A,B,C,D}, with R~{F,FP,B,BP}. Then 
L (C)CL~Q. 
Proof Let z = X IX 2...X~ = > *S. Let 
w = wl...w. be a corresponding morpheme string. To 
differentiate between the occurrence of a symbol as a head 
and otherwise, write C/A/C/B = CA -1C-1B-1' 
S /A /C /B = SA-1C-1B -1 and C /D = CD -1. For 
any rule system R, a redex is two adjacent categories, 
the tail of one matching the head of the other, and is 
reduced to a single category after cancelling the matching 
symbols. Since all occurrences of A must cancel to yield 
a reduction to S, #A = #A -1. This holds for all 
atomic categories except S, for which #S = #S-l+l. 
This lexicon has the property that any derivable 
category symbol, either has exactly one S and is S- 
headed or does not have an occurrence of S. Hence in x, 
#S = 1, i.e., w has exactly one e. Let the number of 
occurrences in x of C/A/C/B and C/D be p and 
q respectively. \]t follows that #C = p +q, #C -1 = p +1. 
Hence q = 1 and w ha.~ exactly one d. Each occurrence 
of C/A/C/B introduces oneA-landB-1. Sincew has 
one e, #A-1 = #B-J = p +1. Hence #A = #B = p +1. 
Since for each A ,B and C in z there must be exactly 
onea,b and c,#a =#b =#c. \[\] 
We show next that in the restricted ease where R 
contains only the two rules FP and B s , the language L 1 
is obtained. 
Lemma 2 Let CG R be the categorial grammar with lexi- 
con FEQ and rule set R = {FP ,Bs }. Then 
L (CGR ) = L1. 
Proof Any x EL 1 has a unique parse of the form 
(Bs FP ) n Bs Bs ~, and hence L 1CL (CG R ). Conversely, 
any x having a parse must have exactly one e. Further, 
all b's and c's can appear only on the left and right of e 
respectively. Any derivable category having an A has the 
form S/(A/)" U where U does not have any A. Thus 
all A's appear consecutively on the left of the e. For the 
rightmost e,F(c) = C/D. A d must be in between a's 
and b's. By lemma 1, #(a)=#(b) =# (c). Thus 
x = a n db n ec" , for some n. Hence L 1 = L (CGR). \[\] 
The next lemma shows that no language intermediate 
to L1 and LEQ can be context-free. It really does not 
involve eategorial grammar at all. 
Lemma 3 If L 1C.L C-LEQ, then L is not context-free. 
Proof Suppose L is context-free. Since L contains 
L1, it has arbitrarily long strings of the form 
a '~ b db"e c". Let k and K be pumping lemma con- 
stants. Choose n >max(K,k). This string, if pumped, 
yields a string not in LEQ, hence we have a contradiction. 
\[\] 
Corollary Let {FP ,Bs }~R. Then there is a non- 
context-free language L ( CGR ). 
Proof Use the lexicon FEQ. Then by lemma 1 
L(CGR)~LEQ. But{FP,Bs}~R,soLI~L(CGR). \[\] 
The following theorem summarizes the results by 
characterizing the rule sets that can be used to generate 
context sensitive languages. 
Main Theorem A categorial system with rule set R can 
generate a context-sensitive language if and only if R 
contains a partial combination rule and a combination rule 
in the reverse direction. 
Proof The "if" part follows for {FP,Bs }by lemmas 
1, 2, and 3. It follows for {BP ,F } by symmetry. For the 
"only if" part, first note that any unidirectional system 
(system with rules that are all forward, or all backward) 
can generate only context-free languages. 5 The only 
remaining cases are {F ,B } and {FP ,BP 1. The first gen- 
erates only context free languages. 5 The second generates 
only the empty language, since no atomic symbol can be 
derived using only these two rules. 
II. CATEGORIAL LANGUAGES ARE PERMUTA- 
TIONS OF CONTEXT-FREE LANGUAGES 
Let VT = {a l, a2 "-.,ak }. A Parikh mapping 6 v/is 
a mapping from morpheme strings to vectors such that 
x~(w) = (#al,#a2 ..... #a k). u is a permutation of v 
iff ~(u)=~(v). Let ~P(L~={W(w)IwEL}, A 
language L is a permutation of L iff ~(L ) = xC(L). We 
define a rotation as follows. In the parse tree for u E L, at 
any node corresponding to a B redex or BP-redex 
exchange its left and right subtrees, obtaining an F-redex 
or an FP-redex. Let v the resulting terminal string. We 
say that u has been transformed into v by rotation. 
We now obtain results that are helpful in showing 
that certain languages eannol be generated by. categorial 
grammars. First we show that, every categorial language 
is a permutation of a context free language. This will 
enable us to show that properties of context-free 
languages that depend only on the symbol counts must 
also hold of categorial languages. 
Theorem Let R c: {F, FP, B, BP}. Then there exists a 
LCF such that ¢(L (CGR)) = ¢(LcF), where LcF is 
context free. 
Proof Let x eL (CGR). In its parse tree at each 
node corresponding to a B-redex or a BP-redex perform 
a rotation, so that it becomes a F -redex or a FP -redex. 
Since the transformed string y is obtained by rearranging 
the parse tree, xt,(x)= ~(y ). Also y derivable using 
R I = {FP ,F } only. Hence the set of such y obtained as a 
permutation of some x is the same as L (CGRt), which is 
context free, 5 i.e., L ( CGR I) = LCF . \[\] 
76 
Corollary For any R ~ {F, FP, B, BP}, L (CGR) is 
semilinear , Parikh bounded and has the linear growth 
property. 
Semilinearity follows from Parikh's Lemma and 
linear growth from the pumping lemma for context-free 
languages. Parikh boundedness follows from the fact that 
any context-free language is Parikh bounded. 6 I-1 
Proposition Any one--symbol categorial grammar is reg- 
ular. 
Note that if L is a semilinear subset of nonnegative 
integers, {a n In eL } is regular. 
III. NON-CATEGORIAL LANGUAGES 
We now exhibit some non-categorial languages and 
compare eategorial languages with others. From the corol- 
lary of the previous section we have the following results. 
Theorem Categorial languages are properly contained in 
the context-sensitive languages. 
Proof The languages {a h (n) \[ n >/0 }, where 
h (n)=n 2 or h (n)=2" which do not have linear growth 
rate, are not generated by any CGR. These are context 
sensitive. Also{arab" I either m>n ,grin is prime and 
n ~<m and m is prime } is not semilinear 7 and 
hence not categorial. 
It is interesting to note that lexieal functional gram- 
mar can generate the first two languages mentioned 
above 8 and indexed languages can generate 
{a nbn2a ~' In>tl}. 
Linguistic Properties 
We now look at some languages that exhibit cross- 
serial dependencies. 
Let G3 be the CGR with R ={FP,Bs}, 
VT = {a ,b ,c ,d }, and with the lexicon 
FFI~I =IS~S1}'= {S lIB/S 1,F(c)={S1}'B }. F(a)=lS1/a/sl, m},Then 
L3 = L (G3) = {wcdw tw E{a,b}*}. The reasoning is 
similar to that of lemma 1. First #c = #d = 1, from 
#S = 1. Since we have Bs rule, c occurs on the left of 
d and all occurrences of a and b on the left of c get 
assigned A and B respectively. Similarly all a and b 
on the right of c, get assigned to the complex category as 
defined by F. It follows that all symbols to the right of 
d get combined by FP rule and those on the left by Bs 
rule. Hence a symbol occurring n symbols to the right of 
d must be matched by an occurrence n symbols to the 
right of the left-most symbol. 
For any k, let G4(k) be the CGR with 
R = {FP ,Bs } again, VT = {al ,hi \] 1 <~ i ~k } U 
{ci I1 ~<i <k} O {d,e}, and the lexicon 
F(b,) ={s,/ai/s,}, F(al) =\[A,},l<~ i <~k, 
F(e,) ={S,/S,+I},I <i < k, F(d) ={Sk}, 
F (e) = {S/S a}. Then 
L (G,(k)) = lal"~a2 "2 --- a~"kdebl"'cx ' ek-~ bk"kJ 
for any k. Note that #A i = #Ai -a. This implies 
#b i = #a i . The rest of the argument parallels that for 
L3 above . Thus {FP, Bs } has the power to express 
unbounded cross-serial dependencies. 
Now we can compare with Tree Adjoining Grammars 
(TAG). s A TAG without local constraints cannot generate 
L3. A TAG with local constraints can generate this, but it 
cannot generate L6 = {am b" c m d" \] m,n >-1}. L4(2) can 
be transformed into L6 by the homomorphism erasing 
ca,d and e. TAG languages are closed under homomor- 
phisms and thus the categorial language L4(2) is not a 
TAG language. TAG languages exhibit only limited cross 
serial dependencies. Thus, though TAG Languages and 
CG languages share some properties like linear growth, 
semilinearity, generation of all context-free languages, 
limited context sensitive power, and Parikh boundedness, 
they are different in their generative capacities. 
Acknowledgements We would like to thank 
Weiguo Wang and Dawei Dai for helpful discussions. 
References 
1. Yehoshua Bar-Hillel, "On syntactical categories," 
Journal of Symbolic Logic, vol. 15 , pp. 1-16 , 1950. 
Reprinted in Bar-Hillel (1964), pp. 19-37. 
2. Haim Gaifman, Information and Control, vol. 8, pp. 
304-337, 1965. 
3. Yehoshua Bar-Hillel, Language and Information, 
Addison-Wesley, Reading, Mass., 1964. 
4. Anthony E. Ades and Mark J. Steedman, "On the 
order of words," Linguistics and Philosophy, vol. 4, 
pp. 517-558, 1982. 
5. Joyce Friedman, Dawei Dai, and Weiguo Wang, 
"Weak Generative Capacity of Parenthesis-free 
Categorial Grammars," Technical Report #86-1, 
Dept. of Computer Science, Boston University, 1986. 
6. Meera Blattner and Michel Latteux, "Parikh- 
Bounded Languages," in Automata, Languages and 
Programming, LNCS 115, ed. Shimon Even and 
Oded Kariv, Springer-Verlag, 1981. 
7. Harry R. Lewis and Christos H. Papadimitriou, Ele- 
ments of the Theory of Computation, Prentice- 
Hall, 1981. 
8. Aravind K. Joshi, "Factoring reeursion and depen- 
dencies: an aspect of tree adjoining grammars and a 
comparison of some formal properties of TAGs, 
GPSGs, PLGs and LFGs," 21st Ann. Meeting of the 
Assn. for Comp. Linguistics, 1983. 
77 
