TWO DISCOURSE GENERATORS 
William C. Mann 
USC Information Sciences Institute 
WHAT IS DISCOURSE GENERATION? 
The task of discourse generation is to produce multisentential text in 
natural language which (when heard or read) produces effects 
(informing, motivating, etc.) and impressions (conciseness, 
correctness, ease of reading, etc.) which are appropriate to a need or 
goal held by the creator of the text. 
Because even little children can produce multieententiaJ text, the task of 
discourse generation appears deceptively easy. It is actually extremely 
complex, in part because it usually involves many different kinds of 
knowledge. The skilled writer must know the subiect matter, the beliefs 
of the reader and his own reasons for writing. He must also know the 
syntax, semantics, inferential patterns, text structures and words of the 
language. It would be complex enough if these were all independent 
bodies of knowledge, independently employed. Unfortunately, they are 
all interdependent in intricate ways. The use of each must be 
coordinated with all of the others. 
For Artificial Intelligence, discourse generation is an unsolved problem. 
There have been only token efforts to date, and no one has addressed 
the whole problem. Still, those efforts reveal the nature of the task, 
what makes it diffic;,It and how the complexities can be controlled. 
In comparing two AI discourse generators here we can do no more than 
suggest opportunities and attractive options for future exploration. 
Hopefully we can convey the benefits of hindsight without too much 
detailed description of the individual systems. We describe them only in 
terms of a few of the techniques which they employ, partly because 
these tschnk:lUes seem more vaJuable than the system designs in which 
they happen to have been used. 
THE TWO SYSTEMS 
The systems which we study here are PROTEUS, by Anthony Davey at 
Edinburgh \[Davey 79\], and KDS by Mann and Moore at ISI \[Mann and 
Moore 801. As we will see, each is severely limited and idiosyncratic in 
scope and technique. Comparison of their individual skills reveals some 
technical opportunities. 
Why do we study these systems rather then others? Both of them 
represent recent developments, in Davey's case, recently published. 
Neither of them has the appearance of following a hand-drawn map or 
some' other humanly-produced sequential presentation. Thus their 
performance represents capabilities of the programs more than 
cs4)abilities of the programmer. Also, they are relatively unfamiliar to 
the AI audience. Perhaps most importantly, they have written some 
of the best machine-produced discourse of the existing art. 
Rrst we identify particular techniclues in each system which contribute 
strongly to the quality of the resulting text. Then we compare the two 
Systems discussing their common failings and the possibilities for 
creating a system having the best of both. 
DAVEY'S PROTEUS 
PROTEUS creates commentary on games of tic.tac-toe (noughts and 
crosses.) Despite the apparent simplicity of this task, the possibilities of 
producing text are rich and diverse. (See the example in Appendix .) 
The commentary is intended both to convey the game (except for 
insignificant variations of rotation and reflection), and also to convey 
the significance of each move, including showing errors and missed 
opportunities. 
PROTEUS can be construed as consisting of three 13rincipal 
processors, as shown in Figure 1. 
Move characterization employs a ranked set of move generators, 
each identified as defensive or offensive, and each identified further 
with a named tactic such as blocking, forking or completing a win. A 
move is characterized as being a use of the tactic which is 
associated with the highest-ranked move generator which can 
generate that move in the present situation• The purpose of move 
characterizaiton is to intefl:ret the facts so that they become significant 
to the reader. (Implicitly, the system embodies a theory of the 
significance of facts.) 
Game Transcript 
in 
I i i=. ASTO=O   l I Move AN0 I' 1 
! I C)~'TP..~MtNATION I I 
SENTENCE 
GENERATION 
Commentary 
out 
Figure 1 : Principal Processors of PROTEUS 
Contrast arises between certain time-adiacent moves and also 
between an actual move and alternative possibilities at the same point. 
For example: 
• Best move VS. Actual move: The move generators are 
used to compute the "best" move, which is compared to 
the actual one. If the move generator for the best move has 
higher rank than any generator proposing the actual move, 
then the actual move is treated as s mistake, putting the 
best move and the actual move in contrast. 
.Threat VS. Block: A threat contrasts with an 
immediately following block. This contrast is a fixedreflex 
of the system. It seems accedteble to mark any goal pursuit 
followed by blocking of the goaJ as contrastive. 
Sentence scope is determined by several heuristic rules including 
I. Express as many contrasts as possible explicitly. (This 
leeds to immediate selection of words such as "but" and 
"however".) 
43 
2. Limit sentences to ,3 clauses. 
3. Put as many clauses in a sentence as possible. 
4. Expmas only the worst of several mistakes. 
The main clause struotum is built before entering the grammar, 
Both the move characterization process and the use of contrasts as the 
principal ~ of sentence scope contribute a great deal to the quality 
of the resuRing text. However, Davey's central concern was not with 
these two 9rocessos but with the third one, sentence generation. His 
system includes an elaborate Systemic Grammar, which he de,scribes in 
datall in \[Devey 79\]. The grammar draws on work of Halliday \[Halliday 
76\], Hudson \[Hudson 71\], Winograd \[Winograd 72\], Sinctalr \[Sinclair 
72\], i~uddleston \[Huddlaston 71\] and F_ K. Brown, following H,..,d_~_n 
most closely.1 
Hudson's work offers a number of significant advantages to anyone 
comddering implementing a discoume generation system. 
I. ComDrehensivaness- Its coverage of English is more 
extensive than comparable work. 
2. Explicitness. the rules are spelled out in full in formal 
notation. 
3. Unity. Since the grammar is defined in a single pubilcalion 
with a single 8uthomhiD, the is*ups of compatibility Of parts 
are minimized, 
It is intemsUng that Oevey does not employ the Systemk: Grlmm~lr 
dehvstJon rules at the highest level Although the grammer is defined in 
terms of the generation of sentences, Devoy entem it at the clause level 
with 8 sents~cs desc~Dtlon whi¢;h conforms to Systemic Grammar but 
was built by other means. A sentence st this level is temporal 
principally of Ctl-_,,~__. but the surface conjunotlens have already 
been chosen. 
Although Oavey real(as no claim, this may redrasent a gener~d result 
about text generation systems. Above some level of al:atnm~on in the 
text planning proces~ planning is not conditioned by the content of the 
grammar. The obvious place to exbeot planning tO become 
indegendertt of the grammar is at the sentence I~. But in both 
PROTEUS and KD.~ Operations independent of the grammar extend 
down to the level of independent clm within sentences. Top leve~ 
coniunctlons am not within such ci~,~__; so they are determined by 
Dlenning pr~ before the grammar is enter~l. 
It would be extremely awkward to implement Oavey'$ sentence s¢obe 
heuristics in a syetamic grammar. The formalism is not well suited for 
oDer~tion~ such as maximizing the total number of explicit contrastive 
(dements. However, the problem is not just a i~rololem with the 
formalism; grammars generally do not deal with this sort of operations, 
and so are ~oorly equil~ped to do so. 
them i~ no need to "rw~nm" it. G~mo~ ~ dlvid~d imo ~ Id~ ~ &¢~Iv~. 
mle-ll3t~lca~nL A sylmlm of choi¢.e~ (surJ1 u t~e ch~¢~ i~lt~men -d~ -ind 
"~mm,e" kT,~,-,~ de.minim *,vh~J~ cm "ai~-ttve') is mech~ ~ other 
cboP~a ~d w is ¢ondi~m¢. but cny ch~¢e. ~¢e mechU, ".. ,.m,~m~rair,~L . Ru~ 
S~lUenc~ ~ femunl-emL ejcn ~ tl~ "We~l." ~ml¢~ enet~le ieed¢~ m,l~¢ltu~ enl n~ tm'efenemmnL 
Although the computer scientist who tries to learn from \[Oavey 79} will 
find that it presents difficulties, the underlying system is interesting 
enough to be worth the trouble. Devey's imDiementation generally 
allam~s to be orthodox, conforming to \[Hudson 71\]. Davey 
regularizes some of the rules toward type uniformity, and thus reduces 
the apparent correspondence to Hudson's formulabons. However, the 
linguistic babe does not appear to have been compromised by the 
implementation. 
One of the major strengths of the work is that it takes advantage of s 
comprehenal~, explicit and linguistically justified grammar. 
Text quality is also enhanced by some simple filtering (of what will be 
expressed) based on demmdencies between known facts. Some facts 
dominate otherJ in the choice of what tO Say. If them is only one move 
on the board having a certain significance, say "threat", then the move 
is described by its significance alone, e.g. "you threatened me" without 
location informatic, n, since the reader can infer the locations. Similarly, 
only the most significant defensive and offensive aspects of a move ate 
described even though all are known. 
The resulting text is divn) and of good quality. Although them ere 
awlo~mrdn __es,~__~ the immense advantage conferred by using a 
sophisticated grammar prevails. 
MANN AND MOORE'S KDS 
Major Modules of KDS 
SOace precJudes a thocou0h description of KDS, but fuller deecriptione 
are mml~ie \[Mann and Moore 80\], \[Mann 79\], \[Moore 7% 
KDS consists Of five me~r modules, as indicated in Figure 2. A 
Frl~lmentM is re~oonalble for eXtnL~ing the relevant knowledge from 
the notation given to it and dividing that knowledge into small 
exl:nmalble units, which we call fragments or pmtosentance¢ A 
Prod=~m Solver, a goal-Oumuit engine in the AI tradition, is responsible 
for seeotlng the I~eUntmlm~d style of me text and ~ for iml~l~ng 
the grol8 ol~glmlze~Ion onto the text accordlng to m8~ style. A 
Knowk~ge Rater removes protasentencas that need not be expressed 
because they would be redundant to the medsr. 
KDS MODULf:S MODULE RESPONSIBILITIES 
i • ExtmcHon of knowledge 
FRAGMENTER from exteffmJ notation 
• D;v~sion ;ere express;bJe clauses 
"P'ROBLEM SOLVER • Style se|ection 
• Gross erc, l'cm;zat|on of text 
KNOWLEDGE FILTER * Cogn;tive redundancy removal 
HILL CLIMBER * Composltion of concepts 
• Sentence quaJ;~ seek|n~ 
e Fermi text ~'m=tion SURFACE SENTENCE 
MAKER 
Figure 2: KDS Module Resgonsibilltiss 
The I~est and moat interesting r~__,_,~e is the Hill Climber, which has 
three raspon~billtis¢ tO compose complex i:rotoasntences from simpM 
one~ tO judge relative quality among the units resulting from 
compo~dtton, and to repeatedly improve the set of protosentencas on 
the Ioasm of those judgments so thM it is of the highest eyeful quality. 
Finally. s very simple Surface Sentence Maker cremes the sentences of 
me final text out of protoaec~lmc~. 
44 
The data flow of these modules can be thought of as a simple pipeline, 
each module processing the relevant knowledge in turn. 
The principal contributors to the quality of the output text are: 
1. The Fragment and Compose Paradigm: The information 
which will be expressed is first broken down into an 
unorganized collection of subsententiai (¢oproximstely 
clause-level) propositional fragments. Each fragment is 
crested by methods which guarantee that it is expressible 
by a sentence (usually a very short one, This makes it 
possible to organize the remainder of the processing so 
that the text production problen~ is treated as an 
improvement problem rather than as a search for feasible 
solutions, a significant advantage.) The fragments are then 
organized and combined in the remaining processing. 
2. Aggregation Rules: Clause-combining patterns of English 
are represented in a distinct set of rules. The rules specify 
transactions on the set of propositional fragments and 
previous aggregation results. In each transection several 
fragments are extracted and an aggregate structure 
(capable of representation as a sentence) is inserted. A 
representative rule, named "Common Cause," shows how 
to combine the facts for "Whenever C then X" and 
"Whenever C then Y" into "Whenever C then X and Y" at s 
propositional level. 
3. Preference Assessment: Every propositional fragment or 
aggregate is scored using a set of scoring rules. The score 
represents s measure of sentence quality. 
4, Hill Ctimbing: Aggregation and Preference Assessment are 
aJternated under the control of a hill-climbing algorithm 
which seek.'s to maximize the overall quality of the 
collection, i.e. of the complete text. This allows a clean 
separation of the knowledge of what could be said from the 
choice of whet should be said. 
5. Knowledge Filtering: Propositions identified by an extolicit 
model of the Reader's knowledge as known to the reader 
are not exl:resasd. 
The knowledge domain of KDS' largest example is a Fire Crisis domain, 
the knowledge of what happens when there is a fire in a computer 
room. The task was to cause the reader, a computer operator, to know 
what to do in all contingencies of fire. 
SYSTEI~ 1 (~OMPARISONS 
The most striking impression in comparing the two systems is that they 
have very little in common. In particular, 
1. KDS has sentence scoring and a quslity.based selection of 
I~ow to say things; PROTEUS has no counterp;u't. 
2. PROTEUS has a sophisticated grammar for which KOS has 
only a rudimentary counterpart, 
3. PROTEUS has only a dynamic, redundancy-based 
P, nowledge filtering, whereas the filtering in KOS removes 
principally St=~tic, foreknown information. 
4. KDS has clause-combining rules which make little use of 
conjunctions, whereas PROTEUS has no such rules but 
makes elaborate use of coniunctions. 
5. KOS selects for brevity above all, whereas PROTEUS 
selects for contrast =hove all. 
6. PROTEUS takes great advantage of fact significance 
assessment, which KDS does not use. 
They have little in common technically, yet both produce high quality 
text relative to predecessors. This raises an obvious question-- Could 
the techniques of the two systems be combined in an even more 
effective system? 
There is one prominent exception to this general lack of shared 
functions and characteristics, Recent text synthesis systems \[Davey 
79\], \[Mann end Moore 80\], \[Weiner 80\], \[Swartout 77\], 
\[Swartoutthesis 81\], all include a facility for keeping certain facts or 
ideas from being expressed. There is an implicit or explicit model of the 
reader's knowledge. Any knowledge which is somehow seen as 
obvious to the reader is suppressed. 
All of the implemented facilities of this sort are rudimentary; many 
consist only of manually-ornduced lists or marks. However, it is clear 
that they cover a deep intellectual problem. Discourse generation must 
make differing uses of what the reader knows and what the reader does 
not know. 
It is absolutely essential to avoid tedious statement of "the obvious." 
Proper use of presupposition (which has not yet been attempted 
computationally) likewise depends on this knowledge, and many of the 
techniques for maintaining coherence depend on it as well. But 
identification of what is obvious to a reader is a difficult and mostly 
unexplored problem. Clearly, inference is deeply involved, but what is 
"obvious" does not match what is validly inferable. It appears that as 
computer-generated texts become larger the need for a robust model of 
the obvious will increase rapidly. 
POSSIBILITIES FOR SYNTHESIS 
This section views the collection of techniques which have been 
discussed so far from the point of view of a designer of a future text 
synthesis system. What are the design constraints which affect the 
possibility of particular combinations of these techniques? What 
combinations are advantageous? Since each system represents a 
compatible collection of techniques, it is only necessary to examine 
compatibility of the techniques of one system within the framework of 
the other. 
We begin by examining the hypothetical introduction of the KDS 
techniques of fragmentation, the explicit reader model, aggregation, 
preference scoring and hill climbing into PROTEUS. We then examine 
the hypothetical introduction of PROTEUS' grammar, fact significance 
assessments and use of the contrast heuristic into KDS. Finally we 
consider use of each system on the other's knowledge domain. 
Introducing KDS teohniques into PROTEUS 
Fragment and Compose is clearly usable within PROTEUS, since the 
information on the sequence of moves, particular 
move locations and the significance of each move 
all can be regarded as composed of many 
incleDendent propositions (fragments of the whole 
structure.) However, Fragment and Compose 
appears to give only small benefits, principally 
because the linear sequences of tic-tac-toe game 
transcripts give an acceptable organization and do 
not preclude many interesting texts. 
Aggregation is also useable, and would appear to allow for a greater 
45 
diverSity of sentence forms than Oavey's Secluential 
assembly torocedures allow. In KDS, and 
presumably in PROTEUS as well, aggregation rules 
can be used to make text brief, in effect, PROTEUS 
already has some aggregation, since the way its 
uses of conjunction shorten the text is similar to 
effects of aggregation rules in KDS. 
Prefei'ence judgment and Hill climbing are interQependent in KDS. 
Introducing both into PROTEUS would appear to 
give great improvement, especially in avoiding the 
long awkward referring phrases which PROTEUS 
i=roduced. The system could detect the excessively 
long constructs and give them lower scores, leading 
to choice of shorter sentences in those cases. 
The Explicit Reader model could also be used directly in PROTEUS; it 
would not help much however, since relatively little 
foreknowledge is involved in any tic-tac-toe game 
commentary/. 
Introducing PROTEUS techniques into KDS 
Systemic Grammar could be introduced into KDS to great advantage. 
The KDS grammar was deliberately chosen to be 
rudimentary in order to facilitate exploration above 
the sentence level. (In fact. KDS could not be 
extended in any interesting way without ulxJrading 
its grammar.) Even with a Systemic Grammar in 
KDS, aggregation rules would remain, functioning 
as sentence design elements. 
Fact significance assessments are also compatible with the KDS 
design. As in PROTEUS they would immediately 
follow aoduialtion of the basic grogositianeL They 
could improve the text significantly. 
The contrast heuristic (and other PROTEUS heuristics) would fit well 
into KDS, not as an a priori sentence design device 
but as a basis for assigning preference. Higher 
score for contrast would improve the text. 
In summary, the principal techniques appear to be completely 
compatible, and the combination would surely produce better 
text than either system alone. 
Exchange of Knowledge Domains 
The tic-tac-toe domain would fit early into KDS` but the KOS 
text-organization Drocesles (not discuased in this I:~ger) would have 
littJe to do. The fire crisis domain would be too complex for PROTEUS. 
It involves several actorS at once, several parallel contingencies and no 
single clear organizing principle. PROTEUS lacks the necessary 
text-organization methods. 
SHARED SHORTCOMINGS 
These systems share (with many others) the i=rimitive state of the 
computer.be,sad discourse-generation a~. Their groce~,~l are 
\[=rimarily devoted to activities that go without notice among literate 
I~eogle. The deeper linguistic and metorical phenomena usually 
associated with the term "discourse" are hardly touched. These 
systems make little attempt at coherence, and they do not respond in 
amy way to the coherence (or lack of it) which they achieve. 
Presupposition, topic, focus, theme, the pro~er role of inference, 
imglicature, direct and indirect Sl:~ech act performance and a host of 
other relevant concepts all go unrepresented. Even wome, the 
46 
underlying conceotual agpars.tus in both systems is extremely adhoc 
and idiosyncratic, severely limiting the I=OSSiblities for using general 
knowledge of the semantics of English. 
Despite these deficiencies, the systems produce relatively smooth 
readable texL They are significant principally as collections of methods 
which may well survive to become l=arts of more competent and 
general.purOose systems of the future. 
CONCLUSIONS 
PROTEUS and KDS are distinct collections of discourse generation 
techniques. There is little overlap between them, and no a~patent 
conflict involved in using the techniques in combination. A system 
which combined the strengths of both would be significantly better than 
either. 
APPENDIX: TEXT SAMPLES FROM THE T~NO SYSTEMS 
KDS: Fire-Alarm Text for Computer Operstors 
Whenever there is a fire, the alarm system is started, 
which soul!de a bell and starts a timer. Ninety seconds after 
• e timer startlk unless the alarm system is cancelled, the 
system calls Wells Fargo. When Wells Fargo is called, they, 
in turn, call the Fire Department. 
When you hear the alarm bell or smell smoke, stop 
whatever you are doing, determine whether or not there is a 
• fire, and decide whether to permit the ala~;m system or to 
cancel it. When you determine whether there is a fire, if 
there is, permit the alarm system, othewvise cancel iL When 
you i~¢mit the alarm system, call the Fire Department if 
possible, then evacuate. When you cancel the alarm system, 
if it is more then 90 seconds since the timer started, the 
system will have called Wells Fargo already, cth~ 
continue what you were doing. 
.°oo°°o~.oo°. ........................ 
PROTEUS: The move sequence 3169547 
Position 4 6 
numbering 
The move=: 
3 3 1 3169 316954 3169547 
The game started with my taking a comer, and you took 
an adjacent one. I threatened you by taking the middle of 
the edge.opposite that and adjacent to the one which 1 had 
just taken but you blocked it and threatened me. I blocked 
your diagonal and forked you. If you had blocked mine, you 
would have forked me, but you took the middle of the edge 
oppoalte the corner which I took first and the one which you 
had just taken and so I won by completing my diagoned. 
References 
\[Oavey79\] 
\[H=liday76\] 
\[Huddleston 71\] 
\[Hudson 71 \] 
\[Mann and Moore 
\[Mann 79\] 
Davey, Anthony. 
Discourse Production. 
Edinburgh University Press, Edinburgh, 1979. 
Krese, G. R. (editor). 
System and Function in Language. 
Oxford University Press, London, 1976. 
Huddleston, R. D. 
The sentence in written English: a syntactic study 
based on an analysis of scientific texts. 
Cambridge University Press, London, 1971. 
Hudson, R. A. 
North Holland Linguistic Series. Volume 4" English 
complex sentences. 
North Holland, London and Amsterdam, 1971. 
80\] 
Mann, William C., and James A. Moore. 
Computer as Author-Results and Prospects. 
Research regort 79-82, USC/Informetion Sciences 
Institute, 1980. 
Mann, William C. and James A. Moore. 
ComDutar Generation of MultiDaregraDh English 
Text. 
1979. 
AJCL, forthcoming. 
\[Moore 79\] 
\[Sinclair 72\] 
\[Swartout 77\] 
\[Swartout 81 | 
\[Weiner 80\] 
\[Winogred 72\] 
Moore, James A., and W. C. Mann. 
A snapshot of KDS, a knowledge delivery system. 
In Proceedings ot the Conference. 17th Annual 
Meeting of the Association for Computational 
Linguistics, pages 51-52. August, 1979. 
Sinclair, J. Moll. 
A course in sl~oken English: Grammar. 
1972. 
Swartout, William. 
A Digitalis Therapy Advisor with Explanations. 
Technical Rel~ort, MIT Laboratory for Computer 
Science, February, 1977. 
Swartout, William R. 
Producing Explanations and Justifications of Expert 
Consulting Programs. 
Technical Report Massachusetts Institute 
Technology/LCS/TR-251, Massachusetts 
Institute Technology, January, 1981. 
Weiner, J. L. 
BLAH, A System Which ExDlains its Reasoning. 
Artificial Intelligence 15:19-48, November, 1980. 
Winograd, Terry. 
Understanding Natural Language. 
Academic Press, Edinburgh, 1972. 
4"/ 

