:Incr en ntal, Event- oneeptua fization.:and 
Natural Language Generation in Monitoring Environments 
Markus GUHE, Christopher HABEL, Heike TAPPE 
Research Group Knowledge and Language Processing (WSV), 
Department of Informatics, University of Hamburg 
Vogt-KGlln-Strage 30 
........... :225.27..iJamburg,.Gerrna~ay, D,22527 - 
{guhe, habel, tappe}@informatik.uni-hamburg.de 
Abstract 
In this paper we present a psycholinguistically 
motivated architecture and its prototypical 
implementation for an incremental conceptu- 
alizer, which monitors dynamic changes in the 
world and simultaneously generates warnings 
for (possibly) safety-critical developments. It 
does so by conceptualizing events and build- 
ing up a hierarchical knowledge representa- 
tion of the perceived states of affairs. If it 
detects a safety problem, it selects suitable 
elements from the representation for a warn- 
ing, brings them into an appropriate order, and 
generates incremental preverbal messages 
(propositional structures) from them, which 
can be taken by a subsequent component to 
encode them linguistically. 
1 Introduction 
Systems that generate natural  descrip- 
tions of what happens in a dynamically changing 
world can be improved substantially by working 
incrementally. Incrementality enhances the overall 
quality of the systems for three reasons: (1) The 
dynamic nature of a continuous stream of input in- 
formation can be handled more directly and, there- 
fore, easier. (2) Incremental systems are capable 
of producing fluent speech, i.e. speech without ar- 
tificial auditory gaps. (3) Parallelism that comes 
with incrementality makes better use of the avail- 
able resources. 
Furthermore, Reiter (1994), who reviews the 
• architecture of some models of natural  
generation, shows that psycholinguistic and engi- 
neering approaches often result in systems, which 
are similar in crucial respects. In this paper we 
ground on two of these common aspects, namely 
the distinction between what-to-say and how-to- 
say (De Smedt, Horacek & Zock, 1996) and the 
use of a pipeline architecture, which divides the 
generation process "into multiple modules, with 
information flowing in a 'pipeline' fashion from 
one module to the next" (Reiter, 1994). Reiter 
states that these architectures do not require mod- 
ules to work in parallel; if parallelism is used one 
has an incremental model, cf. De Smedt & Kern- 
pen (I 987), Ferreira (1996). 
The primary research topic of the ConcEv ~ 
project is the what-to-say component, in which the 
content of utterances is planned (Reiter's content 
planning, in contrast to the  specific sen- 
tence planning component). We use the terminol- 
ogy of Levelt (1989), who calls the first 
component the conceptualizer, the second the 
formulator. These modules interact via preverbal 
messages, which are propositional, non-verbal 
representations of the utterance built up by the 
conceptualizer. They are transformed by the for- 
mulator into linguistic structures for spoken or 
written output. Besides considering high level 
communicative goals (macroplanning), which are 
in the focus of most computational approaches to 
the what-to-say component, e.g. De Smedt & 
Kempen (1987), McKeown (1985), Hovy (1993), 
Chu-Carroll & Carberry (1998), Radev & McKe- 
own (1998), the type of information to be verbal- 
ized also determines the processes of 
conceptualization on the level of microplanning, 
cf. Levelt (1989). Thus, the traditional top-down 
approaches have to be combined with bottom-up 
data-driven approaches of text planning (Marcu, 
1997~. The conceptualizer that is described in de- 
tail in section 3 fits the pipeline architecture on a 
coarse level, but integrates on finer levels the 
ideas of functional modules (Cahill et al., 1999). 
In the present paper we focus on the task of 
I ConcEv (Conceptualizing Events) is supported by the 
DFG (German Science Foundation) in the priority pro- 
gram 'Language Production' under grant Ha-1237/10 
to Christopher Habel. 
85 
generating verbal descriptions of continuously in-. 
coming input from a changing physical world (see 
section 2, for similar settings cf. Neumann & No- 
vak (1983) and Andr6, Herzog & Rist (1988)). 
This specific task requires an incremental pipeline 
architecture--as there are certain steps that have 
to be carried out in a specific order--and, addi- 
tionally, these steps can be organized in such a 
.... insights .into _psy.cho.lJnguistic.aspects _of natural 
 processing. Our implementation thus 
simulates aspects of behavior, e.g. the effects dif- 
fering time pressure has on verbalizations. 
2 Conceptualizing Events 
If the system has to produce descriptions about 
what it 'sees', the main conceptual task is building way that a sequence of clear-cut modules arises, 
in aralle\[ .P~allel rocessin has up conceptual entities representing spatio- • .which can work ..p ..... :~ " " " g -~ ......... ~eml6oral- constellations -of'theexternai world, i:e. 
the advantage that several tasks can be done si- 
multaneously; thus, while utterances are generated 
for some input, subsequent input can already be 
taken in and processed. 
Simultaneous conceptualization can be used as 
the basis of systems producing verbal messages 
when they detect a (possibly) safety-critical de- 
velopment while monitoring a safety-critical sys- 
tem, like intensive care units, nuclear power 
plants, or airports. A module for the generation of 
natural  can be an effective enhancement 
for monitoring for mainly two reasons: first, in 
most cases operators are busy observing many 
displays. Here the auditory presentation of infor- 
mation can make use of idle cognitive resources of 
the operators and, thus, reduce their workload in 
directing their attention to a development that may 
lead to hazardous situations. 2 Second, the essential 
piece of information can be extracted from a 
highly complex set of multimodal information and 
presented by the system in a crisp way. Language 
is the best conceivable means to transfer informa- 
tion as pointedly as possible. Moreover, taking the 
dynamics of the permanently changing world into 
account has the advantage that safety-critical 
situations can be anticipated earlier and much 
more reliably. Conventional systems, in contrast, 
just compare actual measurements with allowed 
values and give a warning or an alarm when a 
violation occurs. But it is more useful, e.g. for a 
nurse if the system tells her that a patient's blood 
pressure is rapidly dropping than that his blood 
pressure is already dangerously low. 
We see the proposed implementation of an in= 
cremental conceptualizer also as a means to gain 
2 The multimodal monitoring environment proposed 
here reflects the division of labor between the compo- 
nents in working memory (Baddeley, 1986), especially 
between the visuospatial sketchpad (VSSP) and the 
phonological loop. Since the observation of multiple 
display units puts a heavy strain on the VSSP, spoken 
natural  as input of critical information would 
use that subcomponent of working memory, namely the 
phonological loop, which is less strained. 
event conceptualization. Events emerge from dy- 
namic input data, which are segmented by the 
conceptual system into meaningful units (Avra- 
hami & Kareev, 1994). They are therefore internal 
representations rather than external entities: "\[...\] 
events arise in the perception of observers" 
(Zacks, 1997). Consequently, a  produc- 
tion system designed for verbalizing what the 
system perceives has to deal with information 
stemming from multiple modalities, e.g. auditory 
and spatial. In particular, a continuous multimodal 
'perceptual stream' has to be translated into dis- 
crete propositional output (preverbal messages) 
that can be encoded linguistically, cf. Levelt 
(1989). To meet such demands, three subtasks 
have to be solved: (1) The input stream has to be 
subdivided into 'perceptual units'; (2)conceptual 
representations have to be built up from these 
'percepts', which (3) have to be combined to pre- 
verbal messages. For the time being, we take the 
input stream to be strictly sequential, but later ver- 
sions of our model will compute simultaneous 
events, as well. 
According to Habel & Tappe (1999) the func- 
tion of the conceptualizer can be subdivided into 
the following processes: segmentation & group- 
ing, structuring, selection, and linearization. The 
first process operates on the (perceptual) input that 
is segmented into meaningful basic units (seg- 
mentation), and--if possible--two or more of 
these units are grouped together to form more 
complex entities (grouping). The structuring pro- 
cess builds .up multilevel hierarchical structures 
from these meaningful basic units. 
To exemplify these steps we use the scenario 
o,f: monitoring the :taxiing of an aircraft, ~iz. the 
movements of an aircraft from the terminal to its 
assigned runway and vice versa. Air traffic con- 
trollers who guide the movements of aircraft on 
the ground (surface movement controllers, SMC) 
have to rely mainly on visual information---either 
looking out of the window of the control tower or 
getting information from a airfield control moni- 
86 
• :tor--and on communicatiom.with ,.the.,aircraft .......... e.vent,,~,and,:on,-,the.,other~,hand of some~groupings..: 
crews. Yet, in some conditions, e.g. in low- The movement from position B to position C, for 
visibility, this method is not failsafe (although re- example, contains--at least--three sub-phases 
liable). It forces the crews to decrease speed--and corresponding to a straight, a curved and a second 
such increases number and duration of de- straight section of the trajectory. These three 
lays--but it also results in greater safety risks) A phases can be distinguished by segmentation, but 
supporting system that monitors the occurrences are combined by a grouping. Furthermore, the 
on the taxiway can mitigate these effects., structuring process has to build up the different 
• phases to form the.TAXI event, cf. Figure 3. 
J .2r._. Lc.L:_,~J-.28. L_: ........ ~".-..The:,~:third,of:~he ~,above.~mentioned: sub- 
c@ @D Echo processes, selection, has two functions: first, it 
-'--\],~,! detects that there is a conflict or a (possibly) 
safety-critical development or situation and de- 
cides that a warning has to be generated. Second, 
it selects the required information for a suitable 
---~A warning or alert. Since the verbal warning can be 
given on different levels of detail, it is necessary 
to select appropriate events from the event hierar- 
chy for further verbalization. On the one hand, it 
Figure I. Monitoring theTaxiingofanAircrafl: Phases of a would not be adequate to produce a general Complex Event 
warning like "Taxiing problem"--except perhaps 
We will demonstrate the workings of our when there is not enough time or no more infor- 
model of an incremental conceptualizer, which mation available at that moment---on the other 
produces natural  messages for the SMC, hand, it would not be suitable to give an in-depth 
with the example depicted in Figure 1. The flight description of each part of the taxiing. Finally, the 
with the number CK-314 shall taxi from the ter- selected items are brought into an appropriate or- 
minal via taxiway Echo to runway 27. The initial de r by the forth process, linearization. 
position of the airplane is A. It then starts to move Before we describe the internal structure of the 
until it reaches position 13 right before a junction conceptualizer in section 3, we want to discuss the 
where it has to stop and wait until the way is clear core idea of 'event conceptualization' in more 
before moving on. It then starts again and contin- detail. The first question to be answered is how a 
ues (C) but its velocity is too high at point D. continuous (perceptual) input stream can be seg- 
Consequently the plane might not be able to mented into separate events. According to the cut 
branch off at the junction, where it is supposed to hypothesis of Avrahami and Kareev (1994, p. 
turn left into runway 27. At that point the moni- 239), "A sub-sequence of stimuli is cut out of a 
toring system generates a warning that the plane is sequence to become a cognitive entity if it has 
in danger of missing the junction. (If the plane in- been experienced many times in different con- 
deed misses the junction, an alert is generated, but texts." This segmentation takes place in the 'eye 
our example does not include this.) of the observer'. Hence, event conceptualization 
For this task two kinds of information have to partly depends on individual as well as on con- 
be available: the planned movements of the plane textual factors. 
and its actual movements. While the former in- The idea of the cut hypothesis implies the ex- 
formation could be handleddirectly by the con- istence of basic events, which ,are the building 
ceptualizer because they are inherently discrete, blocks in our experience used to trigger segmen- 
the latter are information about a continuously tation. They are minimal conceptual entities hu- 
changing world. Here the perpetual continuous in- • : ~ .man observers ~ascribe a-beginning and,an:end to. 
put stream has to be transformed into discrete Thus, they are perceptual wholes--although they 
items. This process consists, on the one hand, of may have an internal structure--, and are there- 
segmentations into discrete units, e.g. a STOP fore the basis for the interface between perception 
and cognition. Basic events can be grouped to- 
gether to form complex events, e.g. assuming that 3 The reports of incidents and accidents of the Austra- the four basic events GRIP THE HANDLE OF A 
lian Bureau of Air Safety. Investigation is a rich source 
of occurrences that should not happen in civil aircraft WINDOW, TURNING THE HANDLE, PULL, and LET 
operations. GO THE HANDLE are perceived, the complex event 
87 
OPENING A WINDOW. can-b¢.buiitaap. Furthermore, 
subsequent events of opening all windows of a 
room can be grouped to AIRING. But events can 
not only be grouped but also segmented: if the 
event OPENING A WINDOW iS perceived, it can be 
segmented into the respective sub-events. We as- 
sume that hierarchical event structures, which are 
based on knowledge about the internal structure of 
prototypical events, e.g. in the format of scripts 
:Schank & Abeison (1977)¢are.~exepresentational 
backbone of event conceptualization, cf, Habel & 
Tappe (I 999). 
3 An Incremental Conceptualizer 
Incremental processing is the 'piecemeal' and par- 
allel processing of a sequential information 
stream. It is a specific kind of parallel processing 
in that the processes have a fixed order, which De 
Smedt & Kempen (1987) describe as a 'cascade of 
processes', in analogy to a water cascade. This 
metaphor means that, for example, the grammati- 
cal encoding--including lexical access--of an 
utterance segment cannot take place until the in- 
formation 'splashes down' from the conceptual 
encoding process. Figure 2 sketches such a cas- 
cade of dependent parallel processes in our model 
of the conceptualizer: The cascade consists of the 
processes construction, selection, linearization, 
and pvm-generation (preverbal-message-genera- 
tion). These processes also constitute a pipeline in 
Reiter's (1994) sense, but they do work in parallel. 
One central parameter of incremental process- 
ing, which is highly relevant for the format of pre- 
verbal messages, is the size of the increments. 
Assume that a description (no warning this time) 
of the turning of the flight number CK-314 into 
taxiway Echo shall be given. This could be done 
by a proposition like turn(ok314, goal(tw-echo)), 
which is a potential increment for a preverbal 
message. Yet, such a proposition would have to be 
built up completely, before the subsequent com- 
ponents can begin forming it into a sentence like 
'Flight CK 314 turns into taxiway Echo.' Hence 
the formulator coulcL not start processing the first 
element, say turn, as soon as it is received from 
the conceptualizer. In:contrast tothis, we opt for 
an architecture, in which the selection of appropri- 
ate lemmas from the lexicon can start for parts of 
a preverbal message, before other entities are built 
up on the preverbal message level. 
As a consequence, the dynamics in incremental 
processing demands a modified notion of prever- 
bal messages. We conceive of them no longer as 
-.,eomplete,.propositions~as~:i,s~mosfly ~t.he~.oase. in 
approaches combining Levelt's ideas with con- 
ceptual semantics--but as sequences of well- 
formed propositional structures~on a.sub-proposi- 
tional level; in logical terminology: predicate 
symbols, functional expressions, terms, etc. The 
incremental formulator SYNPHONICS, which takes 
specific .well-formed parts of propositions as in- 
put, follows these principles (Abb et al. 1996). 
~..J \[ Chanoe ~-- ~Search 
I ~ in CCR r I ...... 
I Add ~ Chanqei .... I I '~ I RpJ~_c:tic~n \] Traveme i uoncem . 
... ....... J 
I CCR 
: ~ { PVM-Generation \] ....... " ,Azcess to 1,J Element 
Travevze Preverbal Messaqes 
Figure 2. Model of an Incremental Conceptualizer 
3.1 Coarse Architecture 
In short, our conceptualizer performs the task 
'Give warnings about (possibly) safety-critical 
developments and situations!' It operates on two 
different input streams: a discrete one, which 
contains the plans for the movements of the air- 
craft on the ground, and a continuous one, which 
originates in the sensors distributed over the taxi- 
way. Since the conceptualizer cannot directly op- 
erate on the continuous input stream, these input 
information must be converted into a stream of 
discrete basic entities, which are basic events in 
this case. In our example a basic event is induced 
by sensoric data sent to the monitoring system. 
e.g. that a particular aircraft passes its position. 
Since the other input stream is already discrete, it 
simply has to be adapted to the required input 
format of the conceptualizer, i.e. it has to be con- 
vetted into basic events, as well. We will neglect 
this process and concentrate on the continuous in- 
put stream. 
..... Based-on Habel :&-T~appe ~(1999) we propose a 
model of the conceptualizer as depicted in Fig- 
ure 2. It consists mainly of four incremental (cas- 
caded) processes that work on the blackboard-like 
current conceptual structure (CCR). At first sight, 
the use of a data structure, to which more than one 
process has access, seems to collide with the no- 
tion of a cascaded information stream. These 
88 
-processes are interdependent,in ~sucha ~way,. how, 
ever, that they indeed behave incrementally; e.g. 
the selection process cannot select anything that 
has not been inserted into the CCR (constructed). 
The CCR can be seen as a shared memory unit 
with a common data structure. A third kind of in- 
formation is needed for a representation of the 
state of affairs: the constellation.of the terminals, 
taxiways, runways, and the participating object(s), 
:are: . 
( 1 ) construction 
(2) selection 
(3) linearization 
(4) pyre-generation 
The first process comprises the processes seg- 
mentation & grouping as well as structuring of 
Habel & Tappe (1999), apart from the segmenta- 
tions that are already done in. the pre-processing 
or, more generally:~the.,spatial~arrmngement,of the 
world and information about objects in it. For ex- 
ample, there is one node that stands for flight CK- 
314, and all the nodes shown in Figure 3 are 
linked to it via an actor relation. Since this type of 
information is not in the focus of the present pa- 
per, we will not discuss it the following. 
In addition to the cascaded processes there is a 
concept lexicon, accessible via a concept matcher: 
these modules, which are called by the construc- 
tion process, find best matches for structures that 
can either be subsumed by a more complex con- 
cept or may represent still incomplete concepts. 
The first is necessary to build up hierarchical 
structures at all. The second is needed for the gen- 
eration of expectations about developments in the 
near future. When, for example, flight CK-314 is 
at position D, the expectation is generated that it 
will go on straight at the next junction or that it 
will be unable to turn left at the next junction 
when keeping the current velocity. 4 On the other 
hand, after the two nodes STARTi and CHPOS~ 
(Figure 3) are constructed, these are given to the 
concept matcher for a subsumption test, which 
consists of trying to match the nodes onto more 
complex concepts. This yields that they can be 
joined together to a MOVE node (MOVEr). Thus, it 
informs the construction process that a STOP event 
(STOPs) will probably occur in the near future, 
which illustrates the second function of the 
matcher: the generation of expectations. (Even the 
last MOVE of a sequence of MOVE events contains 
a STOP event, because aircraft stop at the begin- 
ning of the runway, which is the last event of the 
taxiing, before they commence the takeoff.) The 
construction process inserts these two new nodes 
together with the information .that the. STOPi node 
is just a hypothesis up to now, nothing actually 
perceived. 
The four cascaded processes that constitute the 
'heart' of the conceptualizer and that will be de- 
scribed in more detail in the following sections 
4 The computation of the velocity is easily, done from 
the sensoric data. 
....... step: ~hes~lectiorr.and thet i nearization processes 
correlate to the ones in Habel & Tappe (1999), 
thus, the first selects nodes for verbalizations, 
while the second brings them into an appropriate 
order. The pyre-generation is an additional proc- 
ess and guarantees that the selection as well as the 
linearization have some time to change (the order 
of) the selected nodes, before they are passed on 
to the formulator. We call this time span the la- 
tency time. 
For the implementation of this architecture and 
(a first version of) the algorithms we use a for- 
malism called referential nets (Habel, 1986), 
which was developed to represent linguistic as 
well as common sense knowledge. Entities are 
represented by referential objects (refOs), which 
can be connected via relations, so that a network 
structure arises. The basic entities the pre-proces- 
sing component produces already contain some in- 
formation about what attributes (e.g. which sort) 
have to be ascribed to a refO. In the following we 
use symbolic constants to refer to refOs. These are 
just arbitrary labels; the important point is that the 
refOs can be related to suitable refOs of subse- 
quent processes, which, for example, stand for 
lexical items. 
3.2 Construction 
The construction process takes basic entities as 
input and builds up a hierarchical knowledge rep- 
resentation of the perceived states of affairs in the 
CCR. In the domain we discuss here, three rela- 
tions are especially relevant for the representation 
of events: (temporal) inclusion. (_), temporal 
precedence (-<), and the match of planned events 
onto actual events (g). For the example described 
.... above::the sub-net- of the:a'eferential: :net ~that con- 
tains the actual events (the ones that have sort A- 
Event) is depicted in Figure 3 (the velocity prob- 
lem is just detected). MOVE2, for example, is tem- 
porally included in the event TAXI (MOVE2 E 
TAXI), the event MOVEt is the temporal predeces- 
sor of MOVE2 (MOVEi -< MOVE?), a matching be- 
tween a planned and an actual event is p.(MOVE~, 
• MOVErs), where MOVt!~ stands for the planned 
89 
TAXl = J 
MOVE 1_................. 
STARTi _ CHPOS~ -- SSTOPi - START2 ~, ~CHPOS2 ~ ~ _..~SqOP2 / 
F b-'.. I F 
Sl S2 $3 S4 $5 $6' S7 S8 59 SIO-SII -SI2 S13 S14 SIS SI6 SI7 518 
-.< -< .< -< -< -< -'< .K -< "< "< -K '< -< < -< -.K 
Figure 3. The Knowledge Representation for the Example (STOP2 is only expected) 
movement from position A to B. 
MOVE is a label for complex events that con- 
sists of maximally three sub-events, namely 
START, CHPOS (CHANGE OF POSITION), and STOP, 
where the first and the last sub-event are optional 
and the middle event can be any kind of move- 
ment along a trajectory. START, CHPOS, and STOP 
nodes contain the sensor data nodes S. Temporal 
inclusion relates also TAXI and the basic events 
and MOVE and basic events, but are left: out in the 
figure to keep it readable. The precedence rela- 
tion, though, exists explicitly only between nodes 
of the same granularity, i.e. between the basic 
events and between the nodes of each intermediate 
level, e.g. CHPOS~ -< STOPi '< START2; the implicit 
precedences have to be derived from these. The It 
relation is not included in Figure 3. We use four 
sorts to subdivide the CCR into four sub-nets, 
each consisting of refOs of one sort: planned event 
(P-Event), actual event (A-Event), problem 
(Problem), and real-world object (R-Object). 
The construction process, which builds up the 
representation of the actually registered events and 
to detect problems, can be realized by the follow- 
ing algorithm 
1. Generate a new node for the basic entity that is pro- 
vided by the pre-processing unit. Link it to other 
nodes according to the information (i.e., attributes) 
coming with the basic entity. 
2. Take the new node, possibly with related nodes-- 
especially those that stand in the ~ relation to this 
one--and hand them over to the concept matcher to 
find the best matching complex concept that con- 
tains these nodes--if there is any~and to find (pos- 
sible) problems. 
3. In case there is a problem, create a new node for it, 
and link it to the involved nodes. Continue with step 
2. (In step 2 the 'new' node is the 'old" one, not the 
problem node!) In case of a complex concept, create 
a new node for it, together with the simpler nodes 
that are still lacking (generation of expectations), 
and link it to the basic nodes it subsumes. 
4. Tr3.' to find relations to other complex nodes with 
which links can be established. 
5. Continue with step 2 to try to find more complex 
concepts and problems, until the next new basic en- 
tity enters the system or until there are no more 
complex concepts. Then proceed with step 1. 
In the following application of this algorithm to 
the example, the It relation is left out until the 
problem is identified. Up to that point, each node, 
except for the S nodes, has a corresponding one in 
the sub-structure of planned events, related by It. 
a. s l is read by the construction process as a basic 
event and inserted into the (up to now empty) 
knowledge representation. (step 1) 
b. This node is handed over to the concept matcher 
(there are no related nodes, yet), which responds 
that this is a START event. (2) 
c. STARTi is generated and linked via E to s~. (3) 
d. Steps 4 and 5 yield no results. 
e. $2, the next basic event, is inserted. A .< relation 
between the two s nodes is established. (1) 
f. Because of their temporal vicinity the two s nodes 
are taken and given to the concept matcher, which 
computes a CHPOS event. (2) 
g. CHPOSt iS generated and linked via .G to S2. (3) (It 
would equally be possible to take $2 and S~ to be 
parts of CHPOSt. The only reason we chose this pos- 
sibility is that it preserves the tree structure--at 
least for the moment.) 
h. STARTi and CHPOSi are linked by -<. (4) 
i. Now STARTi and CHPOS~ are given to the concept 
matcher, which joins them together in a complex 
event MOVE (5, 2). 
MOVE1 and.the 'expectation' .node STOPi are gener- 
ated and MOVEi iS linked via m to its elements (3). 
Step 4 yields no result. 
.MOVE1 is just,part of a-more:complex event TAXI. 
(5, 2) 
TAXI is inserted and linked via E to MOVEi. (3) 
Node Ss is read and subsumed to CHPOS~. In this 
step the attributes of CHPOS~ are updated. (Espe- 
cially interesting for our example: the velocity at- 
tribute.) 
o. Nodes 54 to s7 are integrated and subsumed to 
CHPOS~. 
j, 
k. 
L 
nl. 
n. 
90 

.essary changes(replacements)in the ,traverse, be- 
cause the pvm-generation already took the ele- 
ments that would otherwise have been replaced. 
After the latency time of a node is elapsed the 
pvm-generation passes it on to the subsequent 
process of the formulator. This means--as we ar- 
gued above--that propositions exist only 'piece- 
: ..... ._ wise'. Complete propositions exist, then, on a 
higher level of abstraction. When the PROBLEMv 
SOCCER, 449-4,54. P.rac. of the 8tttECAL Munich. 
Australian Bureau of Air Safety Investigation. 
http://www.basi.gov.au 
Avrahami, J. & Kareev, Y. (1994) The emergence of 
events. Cognition, 53,239-261. 
Baddeley, A. (! 986) Working Memory. Oxford: Oxford 
University Press. 
Cahill, L.; Doran, Ch.; Evans, R.; Mellish, Ch.; Pava, 
D.; Reape, M.; Scott, D. & Tipper, N. (1999) In 
search of a reference architecture-for NLG systems. 
EWNLG-1999, Toulouse. 
node is sel ected.as~ a~pr~.x~er.bal ~'messages~informa~ ~,~v.:.~.~:~3hu~Catrolk. J:t~O.~(2arberazy:,.:S,.~( 1998 ) ~o|taborative. re- 
tion about what is the problematic event (moving 
too fast), the object involved (the aircraft with the 
flight number CK-314) and about the location of 
the movement (taxiway Echo) have to be con- 
veyed. This is done by handing on not only the 
PROBLEMv node but also further nodes that contain 
information about the event (in the CHPOS2 node), 
the flight (in a FLIGHT node), and about the loca- 
tion (in a TW-ECHO node). Thus, the pvm- 
generation considers some important relations to 
other nodes before the PROBLEMv node is passed 
on, and checks what other nodes are needed for 
the verbalization and passes them on, as well. 
The formulator can establish interrelations 
between the refOs by the ascribed relations, e.g. 
the PROBLEMv node contains an actor relation to 
the FLIGHT node, which enables the formulator to 
look up all necessary information at the relevant 
nodes. Taken together they are now equivalent to 
a proposition like problem(ck-314, veloc- 
ity(tooHigh), tw-Echo, goal(rw27). 
4 Conclusion 
We presented a psycholinguistically motivated ar- 
chitecture of an incremental conceptualizer to- 
gether with some remarks on its prototypical 
implementation and how this implementation can 
be used for monitoring purposes. The conceptual- 
izer watches dynamic changes in the world and 
generates on-line propositional, preverbal struc- 
tures that can serve as input to a subsequent com- 
ponent, which encodes these structures linguisti- 
cally. 

References 
Abb, B.; Gfinther, C.; Herweg, M.; Lebeth, K:; Maien- 
born, M. & Schopp, A. (1996) Incremental gram- 
matical encoding--an outline of the SYNPHONICS 
formulator. In G. Adorni & M. Zock, eds., Trends in 
natural  generation: An artificial intelli- 
gence perspective, 277-299, Berlin: Springer. 
Andr6, E.; Herzog, G. & Rist, T. (1988) On the simul- 
taneous interpretation of real world image sequences 
and-their natural  description: The system 
sponse generation in planning dialogues. Computa- 
tional Linguistics, 24, 355-400. 
De Smedt, K.; Horacek, H. & Zock, M. (1996) Archi- 
tectures for natural  generation: Problems 
and perspectives. In G. Adorni & M. Zock, eds., 
Trends in natural  generation, Berlin: 
Springer. 
De Smedt, K. & Kempen, G. (1987) Incremental sen- 
tence production: Self-correction and coordination. 
In G. Kempen, ed., Natural  generation, 
365-76, Boston: Martinus Nijhoff. 
Ferreira, F. (1996) Is it better to give than donate? 
Syntactic flexibility in  production. Journal 
of Memory and Language, 35,724-755. 
Habel, Ch. (1986) Prinzipien der Referentialitdt: Un- 
tersuchungen zur propositionalen Reprdsentation 
yon Wissen. Berlin: Springer. 
Habel, Ch. & Tappe, H. (1999) Processes of segmenta- 
tion and linearization in describing events. In R. Kla- 
bunde & C. von Stutterheim, eds., Representations 
and Processes in Language Production, 117-153, 
Wiesbaden: Deutscher Universit~ts-Verlag. 
Hovy, E. H. (1993) Automated discourse generation 
using discourse structure relations. Artoqcial Intelli- 
gence, 63,341-385. 
Levelt, W.J.M. (1989) Speaking: From intention to ar- 
ticulation. Cambridge, MA: MIT Press. 
Marcu, D. (1997) From local to global coherence: A 
bottom-up approach to text planning. Proc. AAAI 97, 
629-635. 
McKeown, K. (1985) Text generation. Cambridge: 
Cambridge University Press. 
Neumann, B. & Novak, H. -J. (1983). Event models for 
recognition and natural . IJCAI-83,724-726. 
Radev, D. R. & McKeown, K. (1998) Generating natu- 
ral  summaries from multiple on-line 
sources. Computational Linguistics, 24,469-500. 
Reiter E. (1994) Has a consensus NL generation archi- 
tecture appeared, and is it psycholinguistically plau- 
sible? IWNLG-1994, J63-170,Kennebunkport, ME. 
Reithinger, N. (1992) The Performance of an incre- 
mental generation component for multi-modal dialog 
contributions. In: R. Dale, E. Hovy, D. R6sner & O. 
Stock, eds.~ Aspects of automated-natural  
generation, 263-276, Berlin: Springer. 
Schank, R. C. & Abelson, R. P. (1977) Scripts, plans, 
goals and understanding: An inquit T into human 
knowledge structures. Hitlsdale: Lawrence Erlbaum. 
Zacks, J. (1997) Seeing the structure in events. Manu- 
script, Stanford University. 
