A LANGUAGE i BIDIRECTIONAL MODEL FOR NATURAL 
PROCESSING 
Gtinter Neumann 
Lehrstuhl fur Computerlinguistik 
Universit~it des Saarlandes 
Im Stadtwald 15, Bau 17.2 
6600 Saarbriicken 11, FRG 
neumann@coli.uni-sb.de 
ABSTRACT 
In this paper* I will argue for a model of 
grammatical processing that is based on 
uniform processing and knowledge sources. 
The main feature of this model is to view 
parsing and generation as two strongly 
interleaved tasks performed by a single 
parametrized deduction process. It will be 
shown that this view supports flexible and 
efficient natural language processing. 
1 INTRODUCTION 
The aspect of bidirectionality has been 
gaining importance since the growing rate of 
research on natural language generation over 
the last years offers us deeper insights into 
this cognitive ability of humans. There are 
theoretical as well as practical reasons for 
adopting bidirectionality. Theoretically, the 
assumption of common knowledge sources 
for both generation and analysis is essential 
for the view of language as "an interpersonal 
medium and an interface to thought" 
(McDonald 1987). From a psychological 
point of view, there is a certain amount of 
empirical evidence for shared processors or 
facilities (Jackendoff 1987): From a system 
engineering view, a bidirectional system 
produces utterances only from that subset of 
language that it is capable to understand. 
Therefore, inconsistencies of the language 
behaviour of the system can be avoided 
(Jacobs 1988). 
A fundamental requirement of a 
bidirectional knowledge base is that it be 
represented declaratively (Appelt 1987). From 
this viewpoint one can distinguish two 
different types of bidirectional natural 
hmguage systems: 
° systems that use uniform knowledge 
sources, but different processes 
• systems that use uniform knowledge 
sources as well as uniform processes 1 
Up to now, systems that are capable of 
analysing and producing language fall into the 
first class, i.e. they use different operations 
for both directions (cf. Hoeppner et al. 1983; 
Busemann and Hauenschild 1988; Allgayer et 
al. 1989) Currently, it is an open question 
what degree of bidirectionality should or 
could be desired (cf. Appelt 1987; Mann 
1987; McDonald 1987; Shieber 1988; Jacobs 
1988). One of the reasons could be that the 
formal specification of some tasks (e.g., the 
determination of content in generation) is 
currently not well understood in order to 
decide whether they could be bidirectional in 
principle. 
But in some research areas uniform 
processing models have been developed that 
are based on formalisms which are well suited 
for uniform irepresentation and processing, 
e.g., Koskenniemi's (1984) two-level model 
of morphology. Recently, there are first 
approaches to uniform architectures for 
grammaticallprocessing (e.g., Shieber 1988; 
Dymetman and Isabelle 1988; Dymetman et 
al. 1990). These architectures are based on 
Pereira and lWarren's (1983) paradigm of 
parsing as deduction. In principle, parsing 
and generation are viewed as a single 
parametrizeddeduction process. 
PROBLEMS OF BIDIRECTIONAL 
GRAMMATICAL PROCESSING 
* Thanks to Klaus Netter, Karel Oliva, Norbert 
Reithinger, Harald Trost and Hans Uszkoreit for 
fruitful discussions about the aspects of the paper's 
contents. 
1Besides these two classes there are also systems that 
use different knowledge sources that are compiled from 
the same source (e.g., Horacek and Pyka 1988) and 
systems that use common basic representation devices 
• (e.g., Lancel et,al. 1988; Neumann and Finkler 1990). 
- 245 - 
Currently developed approaches that 
consider parsing as well as generation (e.g., 
Shieber 1988; Shieber et al. 1990; Dymetman 
et al. 1990; van Noord 1990; Zajac and Emele 
1990) assume: 
° that both tasks take place independently 
from each other, i.e. an utterance is either 
generated or parsed and 
• that grammatical processing can be 
performed without considerations of 
discourse. 
A great problem with this view is that it offers 
no solution of the problem of choice between 
paraphrases in generation: The proposed 
approaches assume - more or less explicitly - 
modularity between the conceptual and 
grammatical component of a natural language 
system. 2 A great advantage of a modular 
design especially for uniform architectures is 
that it is possible to view the grammatical 
component as relatively autonomous and self- 
contained (cf. Appelt 87). 
But then the following problems emerge: 
The conceptual component will be unable to 
exactly specify the logical form as input to the 
grammatical component that will precisely 
lead to the utterance that reflects the intended 
meaning unless the conceptual module has 
detailed information about the grammar and 
knows when to use a specific construction 
(which renders the modular design 
meaningless). 
On the other hand, when parsing and 
generation are performed within the 
grammatical component by a single process 
only then the opposite view of computing all 
possible parses of an utterance is the 
computation of all possible paraphrases of a 
logical form. When gramm~ttical processing 
should be modelled by means of a 
bidirectional grammar, the declarative 
structure of the grammar must not contain 
pragmatical or stylistical information because 
of the modular design. But then the process 
can only choose randomly between 
paraphrases during generation and this means 
that the intended meaning will possibly not be 
conveyed. 
Ideally, a logical language would be 
helpful which necessarily and sufficiently 
represents all meaning distinctions of natural 
2By a conceptual component I mean either the what- 
to-say component of a generation system or the 
component that performs inference, plan recognition 
or anaphora resolution of an understanding system. 
language. But as Shieber (1988) states "this 
... is just the central problem of knowledge 
representation for natural language 10 
general". Currently, there exist only 
approximate solutions to this problem for 
example the use of canonical logical forms 
(cf. Shieber 88). 3 But this still offers no 
solution of the problem of choice between 
paraphrases. 
In this paper it will be argued that the 
following two points will contribute to an 
approximate solution: 
• interleaved parsing and generation 
° using the language use of interlocutors as 
an additional access criterion to linguistic 
knowledge 
Interleaved parsing and generation means that 
both tasks take place in parallel (see section 
2). In principle this results in a bidirectional 
and incremental flow of information during 
natural language processing (see section 4.1). 
An important point during the use of language 
is that the Choice of linguistic material is 
influenced by the language use of others (see 
section 3). This leads to more flexibility: not 
all necessary parameters (e.g., pragmatical 
values) need to be specified in the input of a 
generator because decision points can also be 
set dynamically during run-time. 
A promising approach to realize these two 
features will be to base grammatical 
processing on a uniform process that is 
parametrized by means of a declaratively 
specified preference structure of knowledge 
sources. But, it is necessary to be aware that 
the grammatical component must be assumed 
to be an integrated part of a whole natural 
language system (in particular in models for 
performing dialogs) in order to realize this 
solution. ; 
Before the architecture of the model will be 
described in section 4 the two issues are 
explained in more detail in the next sections. 
2 INTERLEAVING GENERATION 
AND ANALYSIS 
The strategy of viewing natural language 
processing as based on a uniform deduction 
process has a formal elegance and results in 
more compact systems. There is one further 
advantage that is of both theoretical and 
practical relevance: uniform architectures offer 
the possibility to view generation and parsing 
as strongly interleaved tasks. By this I mean 
3It is questionable whether there exists a full solution. 
- 246 - 
that during performing one task (e.g., 
generation) the other one (e.g., analysis) is 
used for monitoring the former. In principle 
this results in a bidirectional and incremental 
flow of information: 
• During the parse of an utterance the 
addressee of the utterance can 
simultaneously start to construct his 
answer. In doing so, partial results of the 
parsing process can be used directly during 
generation (e.g., if a paraphrase will be 
generated). In such flow of control it will 
be possible that generation can be used for 
completing the resulting structure of 
elliptic, underspecified or ill-formed input 
during the process of understanding or for 
generating paraphrases in due time. 
• During generation interleaved parsing could 
help to avoid the construction of 
ambiguous utterances. E.g., it is necessary 
for a natural language help system to 
generate utterances that reflect exactly the 
intended meaning (if possible at all) to be 
sure that the dialog partner will perform the 
correct operations. For instance, producing 
the utterance "Remove the folder by means 
of the system tools" is better than "Remove 
the folder with the system tools" because 
for the latter utterance there exists the 
reading "Remove the folder that contains 
the system tools", too. 
Of course, it is also possible to analyse a 
generated utterance if processes are 
performing their tasks in an isolated way. 4 In 
such flow of control the complete istructure 
has to be generated again if ambiguities are 
detected that have to be avoided. BeCause the 
source of an ambiguous utterance is not used 
directly to guide the generation process it is 
possible that the newly generated structure is 
still ambiguous (and it may happen that the 
same ambiguous structure is generated again). 
This results in inefficient systems because in 
general the loop between the i isolated 
processes must be performed several' times. 
The advantage of a uniform architecture is 
that intermediate results of one direction can 
4For example, the complete structure of a produced 
utte~mce is analysed during \[he 'anticipation-feedback- 
loop' of the HAM-ANS system (see Hoeppner et al. 
1983) to determine whether it can be actually uttered 
elliptic or not. 
immediately be used in the opposite direction 
to determine the ambiguous information in 
due time. 
3 BIDIRECTIONALITY SUPPORTS 
FLEXIBLE AND EFFICIENT 
GENERATION 
One of the disadvantages of currently 
developed generation systems is that they 
view the structure of linguistic knowledge 
only statically. If alternatives exist for a 
particular linguistic expression, decision 
points are evaluated to determine the 
appropriate actual utterance. It is necessary to 
specify corresponding decision points for all 
possible utterances otherwise the choice must 
be performed randomly (the determination of 
the appropriate set of decision points is one of 
the sources of complexity in existing 
generation systems). The flexibility of such 
systems depends directly on the flexibility that 
is brought into the system via the decision 
points that are specified by hand during the 
development of a generation system (i.e. the 
flexibility is restricted). 
On the other side, in a bidirectional system 
the resulting structures of the parsing task can 
be used directly during generation. E.g., in 
general a set :of alternative lexemes is specified 
during the process of lexical choice which are 
synonymous in the actual situation or when 
the semantic input cannot be sufficiently 
specified (e!g., in German, some drinking- 
devices can be denoted either 'Tasse' (cup) or 
'Becher' (mtip) because their shape cannot be 
interpreted Unequivocally). An appropriate 
choice would be to use the same lexeme that 
was previously used by the hearer (if no other 
information is available). In principle this is 
also possible for the choice between 
alternative syntactic structures. 
This means that uniform architectures offer 
the possibility to model the assumption that 
during communication the use of language of 
one interlocutor is influenced by means of the 
language use of the others. This adaptability 
to the use of language of partners in 
communication is one of the sources for the 
fact that the global generation process of 
humans is flexible and efficient. Of course, 
adaptability is also a kind of co-operative 
behaviour. This is necessary if new ideas 
have to be expressed for which no mutually 
known linguistic terms exist (e.g., during 
communication between experts and novices). 
In this case adaptability to the use of language 
247 - 
of the hearer is necessary in order to make 
possible that the hearer will be able to 
understand the new information. 
I do not want to argue that all choices are 
determined by means of language use of 
others. But, when structures that are 
determined during analysis are considered 
during generation, the number of decision 
points or parameters which have to be 
specified during the development of a 
generation system is reduced. This leads to 
more flexibility: not all necessary parameters 
need to be specified in the input of a 
generator because decision points can also be 
set dynamically during run-time. 
This dynamic behaviour of a generation 
system will increase efficiency, too. As 
McDonald et al. (1987) define, one generator 
design is more efficient than another, if it is 
able to solve the same problem with fewer 
steps. They argue that "the key element 
governing the difficulty of utterance 
production is the degree of familiarity with the 
situation". The efficiency of the generation 
process depends on the competence and 
experience one has acquired for a particular 
situation. In such situations the generation 
process performs its task by using compiled 
knowledge and preferences. 
Currently, it is a great problem how 
compiled knowledge is acquired dynamically 
and how it is activated in particular situations. 
But a uniform architecture as proposed in this 
paper seems to be a promising basis for 
designing such a system, because the 
structures determined during analysislcould be 
used for restricting the potential search space. 
4 AN OUTLINE OF A 
BIDIRECTIONAL ARCHITECTURE 
If both aspects - interleaving parsing and 
generation and using the language use of 
interlocutors as additional criterion for the 
structure of linguistic knowledge - are realized 
within a uniform architecture thenthis will 
increase flexibility and efficiency in natural 
language processing. E.g., when starting the 
generation from a :logical form, the 
grammatical process is able to ::call the 
conceptual module's attention if a subphrase 
causes ambiguity. Thus it is not necessary that 
the conceptual module has detailed 
information about the grammar. 
The flow of control within a system based 
on an interleaved approach is bidirectional. 
E.g., during the generation of an utterance 
partial structures are analysed to avoid 
unnecessary ambiguities. The bidirectional 
flow of control supports incremental 
processing: it is possible to start processing of 
partial structures before the whole structure is 
known. In Finkler and Neumann (1989) and 
Neumann and Finkler (1990) we have already 
described an implemented generation system 
(named POPEL-HOW) that realizes an 
incremental and bidirectional flow of control 
based on a uniform parallel processing model. 
The incremental and bidirectional flow of 
control has two main advantages during 
generation. Firstly, the determination of 
contents can be done on the basis of 
conceptual considerations only, because 
POPEL-HOW is flexible enough to handle 
underspecified input. Secondly, the 
conceptualizer has to regard feedback from 
POPEL-HOW during the computation of the 
further selection process. This means, an 
incremental system like POPEL can model the 
influence of linguistic restrictions on the 
process that determines what to say next. 
Underspecified structures are analysed in 
POPEL-HOW at each level of description by 
means of declarative described mapping rules. 
The analysis of such structures is performed 
with generation specific operations. If the 
system would be based on a uniform 
architecture then such specific operations are 
no more necessary. 
4.1 BiLD - A MODEL FOR 
BIDRECTIONAL LINGUISTIC 
DEDUCTION 
At the University of Saarbriicken a project 
called BiLD is now being started where it will 
be investigated how interleaving of parsing 
and generation can be efficiently performed 
and how such a model can be used for 
increasing flexibility and efficiency during 
natural language processing. Fig. 1 (next 
page) shows the schematic structure of its 
architecture. 
The core of the system is a uniform 
parametrized deduction process. The main 
task for the process in both directions is the 
determination of the corresponding syntactic 
informationi'that functions as an interface 
between graphematic and semantic 
information (a formalism based on Head- 
driven Phrase Structure Grammar (Pollard 
and Sag 1987) will be used). 
- 248 - 
lemllntlo 
exprelsiOn 
I T 
linguistic 
deduction 
process 
I°'n::.:r: '- 
I and I 
Ioxloon 
complied 
ilruolurel 
LI I I lit IrlOll 
Fig. 1 : Schematic structure of BiLD 
The task of the deduction process during 
generation is to construct the graphematic 
form of a specified semantical feature 
description. 5 For example, to yield the 
utterance "A man sings" the deduction process 
gets as input the semantic feature structure 
I \[rel : sing' 
sem" | \[quant : exist' 
"/ /vat:El ,- |agenS:|restr \[pred : man| 
L t :tvar: l \] 
and deduces the graphematic structure 
\[graph : (A_man_sings.) \] 
by means of successive application of lexical 
and grammatical information. In the same way 
the deduction process computes from the 
graphematic structure an appropriate :semantic 
structure in parsing direction. 
The author has now started to develop and 
implement a first version of a prototype of a 
uniform algorithm for HPSG. The main idea 6 
is that the approach is head-driven in both 
directions. In the first phase of the algorithm 
the maximal projection for all head elements 
are computed (or predicted) bottom-up. 
Phrases are then combined top-down. The 
completion step is controlled by syntactic and 
semantic information inherited from lexical 
heads and by the principles of HPSG. 
5The resulting structure of the generation :process as 
well as the input structure of the parsing process is 
written language, therefore we use the feature 'graph' 
instead of 'phon' which is preferably used in Pollard 
and Sag (1987). 
6Basic ideas of the approach are influenced by the 
head-driven parser of Proudian and Pollard (1985). 
Because heads are processed first the 
completion of structures must be performed in 
left as well as in right direction. 
The approach supports the ID/LP format 
of rules. But it is an open question whether 
linear precendence can be processed in the 
same way for generation and parsing. The 
problem is that during parsing the task of LP 
rules is to filter out ungrammatical structures. 
During generation the task of LP rules can be 
seen as an ordering criterion. But in this case 
the problem of choice between paraphrases 
emerges. In POPEL-HOW it is assumed that 
the order of activation of concepts (which is 
determined using pragmatical knowledge) 
should be maintained if it is syntactically 
wellformed; otherwise the segments are 
reordered. Whether such viewpoint is 
acceptable for generation in general is still 
open. 
4.2 ASPECTS OF CONTROL 
STRUCTURE 
A major aspect of the BiLD project is that 
specific parametrization of the deduction 
process is represented in the lexicon as well 
as in the grammar to obtain efficient structures 
of control (Uszkoreit 1991). The main idea is 
that preference values are assigned to the 
elements (disjuncts or conjuncts) of feature 
descriptions. For example, in HPSG all 
lexical entries are put together into one large 
disjunctive form. From a purely declarative 
point of view these elements are unordered. 
But a preference structure is usrd during 
processing in order to guide the process of 
lexical choice efficiently which itself 
influences the grammatical process. 
To support flexibility and efficiency (in the 
way described in section 3) the language use 
of interlocutors will be considered to influence 
the preference values. For example, the 
frequency of access of a lexeme will increase 
its preference value. In a uniform lexicon it is 
no matter whether the lexeme was accessed 
during parsing or generation. But this means 
that the use of particular linguistic elements of 
the interlocutor influences the choice of lexical 
material during generation. 
5 CONCLUSION 
In this paper it is argued that generation 
and parsing should be best viewed as two 
interleaved tasks based on a single 
parametrized deduction process and that this 
view supports flexible and efficient natural 
- 249, 
language processing. A major point of view is 
that the language use of interlocutors should 
be considered during generation as an 
additional access criterion. 
REFERENCES 
Allgayer J.; Jansen-Winkeln R.; Reddig C. and 
Reithinger N. 1989 "Bidirectional use of knowledge 
in the multi-modal NL access system XTRA", 
Proceedings of the l l th International Joint Conference 
on Artificial Intelligence, 1492-1497. 
Appelt, D. E. 1987 "Bidirectional Grammars and 
the Design of Natural Language Generation 
Systems," In Y. Wilks (ed.) Theoretical Issues in 
Natural Processing-3, New Mexico State University, 
Las Cruces, New Mexico, 185-191. 
Busemann, S. and Hauenschild, C. i988 "A 
Constructive View of GPSG or How to Make It 
Work," Proceedings of the 12th International 
Conference on Computational Linguistics, 77-82. 
Dymetman, M. and Isabelle, P. 1988 "Reversible 
Logic Grammars for Machine Translation," 
Proceedings of the 2nd International Conference on 
Theoretical and Methodological Issues in Machine 
Translation of Natural Language. 
Dymetman, M.; Isabelle P. and Perrauit, F. 1990 
"A Symmetrical Approach to Parsing and Generation," 
Proceedings of the 13th International Conference on 
Computational Linguistics, 90-96. 
Finkler, W. and Neumann, G. 1989 "POPEL- 
HOW: A Distributed Parallel Model for Incremental 
Natural Language Production with Feedback," 
Proceedings of the Eleventh International Joint 
Conference on Artificial Intelligence, 1518-t523. 
Hoeppner, W.; Christaller, T.; Marburger, H.; 
Morik, K.; Nebel, B.; O'Leary, M. and Wahlster, W. 
1983 "Beyond Domain-Independence: Experience with 
the Development of a Gel'man Language Access 
System To Higly Diverse Background Systems," 
Proceedings of the 8th International Joint Conference 
on Artificial Intelligence, 643-646. 
Horacek, H. and Pyka, C. 1988 "Anweiadbarkeit 
von Unifikationsgrammatiken ftir effizientes 
Generieren," In H. Trost (ed.) 4.0sterreichische 
Artificial-lntelligence-Tagung, Springer, Berlin, 171- 
177. 
Jackendoff, R. 1987 "Consciousness and the 
Computational Mind," Cambridge Massachussetts: 
MIT Press. 
Jacobs, P. S. 1988 "Ach!eving Bidirectionality," 
Proceedings of the 12th International Conference on 
Computational Linguistics, 267-274. 
Koskenniemi, K. 1984 "A General Computational 
Model for Word-Form Recognition and Production," 
Proceedings of the lOth International Conference on 
Computational Linguistics, 178-181. 
Lancel, J.M.; Otani, M.; Simonin, N. and 
Danlos, L. 1988 "SAGE: a Sentence Parsing and 
Generation System," Proceedings of the 12th 
International Conference on Computational 
Linguistics, 359-364. 
Mann, W. C. 1987 "What is Special About 
Natural Language Generation Research?," In Y. 
Wilks (ed.) Theoretical Issues in Natural Processing- 
3, New Mexico State University, Las Cruces, New 
Mexico, 206-211. 
McDonald D. D. 1987 "No Better, but no Worse, 
than People," In Y. Wiiks (ed.) Theoretical Issues in 
Natural Processing-3, New Mexico State University, 
Las Cruces, New Mexico, 200-205. 
McDonald, D. D.; Meteer, M. W. and 
Pustejovsky, J. D. 1987 "Factors Contributing to 
Efficiency in Natural Language Generation," In G. 
Kempen (ed.) Natural Language Generation: New 
Results in Artificial Intelligence, Psychology and 
Linguistics, Dordrecht: Martinus Nijhoff, 159-182. 
Neumann, G. and Finkler, W. 1990 "A Head, 
Driven Approach to Incremental and Parallel 
Generation of Syntactic Structures," Proceedings of 
the 13th International Conference on Computational 
Linguistics, 288-293. 
van Noord, G. 1990 "Reversible Unification Based 
Machine Translation," Proceedings of the 13th 
International Conference on Computational 
Linguistics, 299-304. 
Pereira, F. C. N. and Warren, D. H. D. 1983 
"Parsing as Deduction, " Proceedings of the 21th 
Annual Meeting of the Association for 
ComputationaL Linguistics, 137-144. 
Proudian, D. and Pollard, C. 1985 "Parsing Head, 
Driven Phrase iStructure Grammar, " Proceedings of 
the 23rd Annual Meeting of the Association for 
Computational Linguistics, 167-171. 
Pollard, C.! and Sag, I. 1987 "Information-based 
syntax and semantics," CLSI Lecture Notes 13, Center 
for the Study of Language and Information, Standford, 
CA. 
Shieber, S. M. 1988 "A Uniform Architecture for 
Parsing and Generation," Proceedings of the 12th 
International Conference on Computational 
Linguistics, 61,4-619. 
Shieber, S.iM.; van Noord, G.; Moore, R. M. and 
Pereira, F. C. P. 1989 "A Semantic Head-Driven 
Generation Algorithm for Unification-Based 
Formalisms," Proceedings of the 27th Annual 
Meeting of the Association for Computational 
Linguistics, 7-17. 
Uszkoreit, H. 1991 "Strategies for Adding Control 
Information to Declarative Grammars," Technical 
Report, Institute for Computational Linguistics, 
University of Saarbriicken, FRG. 
Zajac, R. and Emele, M. 1990 "Typed Unification 
Grammars," Proceedings of the 13th International 
Conference onComputational Linguistics, 293-298. 
- 250 - 
