COLING 8~ Z Horeckp fed.) 
No~h-Hol~ Pub~hi~ ~mpany 
© Academ~ 19~ 
ON THE ROLE OF THE HIERARCHY OF ACTIVATION IN 
THE PROCESS OF NATURAL LANGUAGE UNDERSTANDING 
Eva Haji6ov~, Jarka Vrbov~ 
Department of Applied Mathematics 
Faculty of Mathematics and Physics 
Charles University 
Prague 
Czechoslovakia 
The elements of the stock of knowledge shared 
.by the speaker and the hearer change their sal- 
ience, in the sense of being immediately access- 
ible in the bearer's memory. The hierarchy of 
salience is argued to be a basic component of a 
mechanism serving for the identification of ref- 
erence. Some of the regularities of this mechanism 
are discussed, the description of which is a nec- 
essary prerequisite of an automatic understanding 
of connected texts. 
i. When discussing the question of what is the main contribut- 
ion of computational linguistics to theoretical linguistics at one 
of the panel discussions at Coling 80 in Tokyo, the panelists unan- 
imously pointed to the due emphasis laid on the function of natural 
language in communication. In this direction, it is known from the 
classical Prague School that the topic/focus articulation belongs to 
the most important aspects connecting the structure of language with 
the conditions of its use. However, this articulation I was not exam- 
ined systematically in most of the main linguistic trends, so that 
the pioneers who are aware of its importance for natural language 
understanding (esp. Grosz, 1977, Reichman, 1978, 1981, McKeown, 1979, 
Hirst, 1981) attempt to find their own ways. 
During the discourse the stock of "knowledge" the speaker as- 
sumes to share with the hearer changes according to what is "in the 
centre of attention" at the given time point. 2 Each utterance has its 
influence on this hierarchy of salience; however, not every mention- 
ing of an object has the same effect. The assumption that the degrees 
of salience constitute a partial ordering is corroborated e.g. by the 
degrees of consciousness characterized by Chafe (1974). 
Let us denote by x ~ an expression x referring to an object a 
that is salient to the -qa- degree n in the stock; since the maximum 
of salience can more easily than other degrees be imagined to be fix- 
ed, we denote it by ~ = ~, reversing the direction of"growth" of the 
degrees; to the left (right) of the arrow we indicate the state 
immediately preceding (following) the utterance of a sentence S in 
which x occurs; x c F (T) stands for "x belongs to the focus (~opic) 
of S";--P(x ) denotes that x is expressed either by a weak pronoun 
or Ts d~le~ed in S, though--present in its tectogrammatical represent~ 
ation (TR);3 with--NP. _(x ) we refer to definite NP's and to such _--~a~e a 
NP's as one of the N, ~o~ (of the) N. 
According to a preliminary empirical investigation it appears 
that:the following rules concerning the degrees of salience obtain 
107 
108 E. HAJI~OVA and J. VRBOVA 
(cf. a first formulation of some of them in Sgall, 1980a, p.240), 
where m, n > ~: 
(i) If P(X_a), then a_n- + a_ n- 
(2) If NP(x a) C \[, then a_ n- ÷ a~ 
(3) If NPdef(~a) C ~, then a_n-- + 1 
(4) If a_n-- + a_ m-, then b_ m-+2 obtains for every object b that is 
not itself referred to in the TR of S, but is immediately 
associated with it (as e.g. teacher or pupil are with 
schoo l, or parents with child, e--t-~..) 
(5) If As for xa, or C0noernin~ x a is present in S, then 
a ~ + a I ~ 
(6) If ~a is not included in the TR of S, not even among the 
associated objects (see Rule (4)), then a ~ + a n-+2 
As was pointed out by Sgall (1967a, p.95f.; 198Ob), the differ- 
ence between n = ~ and n = 1 is too small to make the reference 
assignment un\[vocal <if--such specific pronouns as Czech ten, German 
der or Engl. the latter are not present): If Charles met PAUL is 
f-o\[lowed by He wore his HAT, we may only guess whether he refers to 
Paul and his to Charles, or vice w~rsa, or whether a third possibili- 
ty is present. However, whenever the difference in the degrees of sal- 
ience is greater, an occurrence of a pronoun or of a definite NP in 
the next utterance regularly refers - as we want to illustrate in the 
sequel - to that object which was referred to as the most salient a- 
mong those that meet the lexical and grammatical conditions of the 
NP or pronoun <number, gender). The two main types of the thematic 
progressions described by Dane~ (1968; 1976) are important here. Also 
Hobbs (1976) specifies a common heuristics for resolving pronouns, 
which says that we should prefer the subject to an NP in the object 
position (in other words, since the subject position in English 
functions as a rule as (a part of) the topic of the sentence, this 
heuristics says that the tJpic is preferably preserved in successive 
sentences). Another possible strategy <Hobbs, 1979) includes infer- 
encing and factual knowledge. 
In our rules some of the relationships between the salience and 
the pragmatic relationships of lexical units are treated only in a 
preliminary way. Thus e.g. Rule (4) or a similar rule should reflect 
also a rather common situation when a mentioning of a particular ob- 
ject (or objects) brings into foreground only a fraction of a set 
of objects that already are in the fore~round of the stock of know- 
ledge. It does not capture the case when the foregrounded object(s) 
belong to a set the activation of which has already faded away under 
the threshold of activation; it seems to be a plausible hypothesis 
that even in such a case the "rest" of the objects are being activat- 
ed beyond this threshol~ by mentioning the foregrounded object(s), 
even if to a very low extent. 
2. Let us illustrate the above rules on a fragment of text; we 
are not concerned with a systematic linguistic analysis of the given 
text here, but we coBcentrate on the changes of activation of the 
elements of the stock of shared knowledge as reflected (and induced3 
by the discourse. We restrict ourselves to objects identified by NP's 
or pronouns and we leave aside actions and their circumstantial 
ON THE ROLE OF HIERARCHY OF ACTIVATION 109 
qualifications. The capital letters denote the intonation centre of 
each clause; for the ease of reference, the clauses are numbered. 
A partial preliminary discussion of such a text was presented in Ha- 
ji~ov~ and Vrbov~ (1982). 
\[I\] The school garden was full of CHILDREN. 
\[2a\] They talked NOISILY, 
\[2b\] but the teachers did not REPROVE them, 
\[20\] because they were so EXCITED. 
\[3\] Outside PARENTS were waiting. 
\[4\] A group of about five parents grouped around the MICROPHONE. 
\[5\] One of them should probably SPEAK, 
\[6\] The teachers were very SERIOUS. 
\[7\] In fact, ALL ADULTS in the garden were serious. 
\[8\] They were dressed in evening DRESSES. 
\[9\] CQncerning the pupils, they had school UNIFORMS. 
\[i0\] The smallest had even snow-white COLLARS. 
\[lla\] The tall on the contrary LAUGHED at them, 
\[llb\] because they pretended not to be INTERESTED. 
\[12a\] Their teachers did not LIKE that, 
\[12b\] while the teachers of the small were evidently PROUD.. 
\[13\] The parents at the microphone discussed something in a WHISPER. 
\[14\] Then one of them, a father, stepped FORWARD 
\[,15~ and SAID: 
\[16\] The Society of Friends of this school is highly satisfied with 
the results of our lowest GRADES. 
\[17\] Then he PAUSED. 
\[18\] Some parents in the audience began to MURMUR. 
\[i9\] The father at the microphone tried to SILENCE them. 
\[20\] He was puzzled. 
\[21\] The praised children immediately started to C~TTER. 
\[22\] One small pupil SCREAMED: 
\[23\] HURRAH! 
\[24\] The teachers of the older pupils made FACES. 
\[25\] The parents who were representatives of the Society wildly 
GESTICULATED. 
\[26\] A mother suddenly FAINTED. 
\[27\] The courageous child stopped SCREAMING. 
\[28\] His classmates ASKED him so. 
\[29\] The least excited seemed the oldest PUPILS. 
\[30\] The boys ' CHEWED 
\[31\] and the girls GIGGLED. 
A simplified list of the most activated objects after the individual 
utterances: 
\[i\] children ~ school garden I parents 2 school 3 pupils 3 
\[2a\] children ~ parents 2 school garden 3 school 5 ... 
\[2b\] children teachers I parents 2 school 3 ... 
\[2c\] either: children ~ teachers 3 parents 2 school 5 o.. 
or: teachers ~ children 2 school 2 parents 4 o.. 
parents ~ children 2 teachers 5(3) school 5 ... \[3\] 
\[4\] microphone ~ parentsl I {~those at the micro) parents21 children 3 
\[5\] parentsl ~ microphone 2 parents22 children 3 ... 
\[6\] teachers I parents! 3 school 3 children 3 microphone 4 parents25 ... 
\[7\] adults ~ garden I teachers 2 parents/2 parents22 school 4 ... 
110 E. HAJI~OVA and J. VRBOVA 
\[8\] dresses # adults ~ teachers 2 ... 
\[9\] uniforms ~ pupils I dresses 2 adults 2 teachers 3 ... 
\[IO\] collars ~ pupilsll(=smallest) pupils21 teachers 3 dresses 4 ... 
\[lla\] pupilsl I pupils21 (=tall) collars 2 teachers 3 dresses 6 ... 
\[llb\] pupils21 pupilsl 3 (or vice versa) teachers 3 ... 
\[12a\] teachersl I <whose?> pupils21 pupilsl 3 <or vice versa> ... 
\[12b\] teachers21(=of small pupils) teachersl 3 pupilsl 3 pupils23 ... 
\[13\] parentsl I teachersl 3 teachers23 pupilsl 3 pupils23 ... 
\[14\] father I parents 3 pupils(children> 3 teachers 5 ... 
\[15\] father I parents 3 pupils 3 teachers 5 
\[16\] pupilsl ~ Society I father I parents 3 ... 
\[17\] father I pupilsl 2 Society 3 parents 3 ... 
\[18\] parents31 father 2 pupils 3 Society 5 ... 
\[19\] father I microphone I parents 3 pupils 3 
\[20\] father I microphone 3 parents 3 pupils 3 ... 
\[21\] childrenl I father 3 parents 3 ... 
\[22\] pupil I children 3 father 5 teachers5... 
\[23\] pupil 3 children 5 father 5 teachers 5 ... 
\[24\] teachersl I children 3 pupil 5 ... 
\[25\] parents41 Society I teachers 3 pupils 3 ... 
\[26\] mother I parents43 Society 3 teachers 3 pupils 3 ... 
\[27\] childl I mother 3 parents 3 teachers 3 pupils 3 Society 5 ... 
\[28\] classmates I childl I mother 3 ... 
\[29\] pupils2 ~ classmates 3 childl 3 mother 3 ... 
\[30\] boys I pupils22 children 3 girls 3 ... 
\[31\] girls I boys 3 children 3 pupils2 9... 
Commentary 
The following comments are numbered according to the number of the 
clause to which they refer. We omit those clauses the commentary to 
which follows trivially from the previous comments and from the rules. 
\[I\] In the opening sentence of the text two objects are introduced: 
the school garden and the children. Since the latter object is ment- 
ioned in the focus of the sentence, it becomes more salient than ob- 
jects mentioned previously or those activated by their presence in 
the broader context or situation. Also the associated objects such 
as school, pupils, and teachers are now relatively salient. 
\[2a\] The children as a highly salient object in the stock of shared 
knowledge are referred to in the topic position by a pronoun, which 
keeps their activation at the same degree (see Rule El)), while the 
activation of the other objects <Rule (6))fades away. 
\[2b\] A new object enters the scene: the teachers. However, their 
appearance on the scene is not completely new and surprising, because 
the hearer has already been introduced into the scene of school, 
which has the teachers associated with it (see Rule <4)). This is 
why the teachers can be understood as referring to the teachers of 
the children and the school which were already mentioned. In accord- 
ON THE ROLE OF HIERARCHY OF ACTIVATION 111 
ance with Rule (3) the degree of salience of teachers is I. The 
children remain on the scene - they are reminded by a pronominal re- 
ference in the topic and their activation does not fade away (see 
Rule <i)). It should be noted that even though the pronoun them is in 
the final position of the sentence, it is preceded by the intonation 
centre; it is well known that all participants placed after the inton- 
ation centre belong to the topic part of the sentence. 
\[2c\] The pronoun in the topic has an indistinct reference: both teach- 
ers and children may be referred to. This indistinctness of reference 
is difficult to solve: according to the strategies mentioned in Sect. 
2 the two objects are salient enough to be referred to by a weak pro- 
noun, and the teachers as well as the children can be excited in the 
given situation, so that not even factual knowledge with inferencing 
gives a safe clue. The formulation of the text can be characterized 
as not successful in this point. However, if the lexical setting of 
\[2c\] were changed, as in \[2c ~\] and \[2c''\], the indistinctness would 
disappear, and only one of the two possible candidates of reference 
would be chosen, namely the teachers in \[2c'\] and the children \[2c''\]: 
\[2c'\] ... because they were always kind to their pupils. 
\[2c''\]... because they were too small to understand the seriousness,.. 
\[3\] A new object is being introduced in the focus of the sentence; a- 
gain, it can be said that it is not completely new, because it is also 
given by the broader scene <of family relations: children - parents). 
This newly introduced item receives a very high degree of activation, 
while the other two objects - the children and the teachers - are not 
mentioned in this sentence at all and thus their activation fades 
away (see Rule <6)). 
\[4\] The speaker chooses a subset of the most activated set of elements 
and speaks about this subset; he introduces a new object on the scene, 
which receives a high degree of activation (see Rule <2)), but since 
it does not appear in any of the following utterances, this activat- 
ion later fades away step by step <Rule (6)>. The complement to the 
set of five parents - the rest of the pa~ents - retains the degree of 
a rather high activation, see Rule (3). 
\[6\] One of the items the activation of which has been fading away 
<the teachers) is back on the scene; the teachers may be referred to 
by a definite NP in the topic since the activation has not been low- 
ered under a certain threshold, which would otherwise lead to an 
explicit reintroduction. 
\[~9\] An explicit reintroduction of the children is preferred by the 
speaker since the children have not been mentioned in several preced- 
ing utterances; their activation has been lowered considerably. For 
such strong re-activation expressions such as concernin@, as far as 
... is concerned, as for ... are used. 
\[I0\] Only one subset of the children is explicitly mentioned in this 
sentence; the activation of this item of knowledge is raised <Rule 
<3)), but also the complement subset of pupils is "supported" in its 
activation <Rule <i)), which does not fade away as it would be the 
case if no pupils were menuioned at all <Rule <6)). 
\[llb\] The reference of the pronoun ~ is again indistinct; it may 
refer either to the smallest or to the tall pupils. Neither of the 
strategies mentioned above helps: both the NP s the tall <pupils) 
and the smallest <pupils) are in the topic position of \[lla\], and 
both are activated to the same extent: the smallest pupils were re- 
ferred to in \[lla\] by pronoun them and the activation'of this item 
was preserved <Rule (I)) ; the tall were mentioned by a definite NP in 
the topic, which keeps their activation. This indistinctness remains 
112 E. HAJI~OV~, and S. VRBOV.~ 
unsolved even after the utterance of \[12a\], which adds another in- 
distinct reference <their teachers - whose? of the smallest? of the 
tall?) and it is only after hearing \[12b\] that the hearer can decide: 
there the explicit mentioning of the teachers of the small children 
resolves the indistinct reference of their in \[12a\] and at the same 
time our factual knowledge of the world invites the inference that if 
the small children pretended not to be interested, their teachers 
would have nothing to be proud of, so that they in \[llb\] can refer 
only to the tall pupils. 
\[16\] The speaker of the direct speech is not mentioned in the quote 
(only by means of his reference to the Society of Friends of the 
School, of which he undoubtedly is a member), but the activation of 
this item does not seem to fade away. This appears to be a general 
feature of direct speech, which follows from the semantico-syntactic 
structure of the direct speech, where the "direct speech clause" 
functions as an object (Patient) of the introductory clause (and the 
speaker, as the Actor of the sentence, belongs to the TR of the same 
sentence). 
\[27\] The use of the courageous child to refer to the small pupil who 
screamed "Hurrah!" offers another example of the inferencing the hear- 
er must perform in order to understand what is the sentence about and 
to determine the reference, even in cases where full NP's rather than 
pronouns are used. 
It is also easy to verify that with other variants of the text 
the weak pronoun they, if it were used after each of the sentences 
\[5\](e.g. in 6": The children observed them, ...) , \[6\], \[13\], \[18\], 
etc., would lack any referential indistinctness; similarly for he 
after \[14\] <where, in fact, the pronoun is deleted),\[17\], \[22\], etc. 
The referential indistinctness is present (and must be solved by 
means of inferencing) after such utterances as e.g. \[2b\], \[9\], \[IO\], 
\[lla\], \[12a\], where objects with the degrees @ and 1 can be referred 
to by the same pronoun. In a similar wav the indistinctness of the 
reference of nouns can be studied,cf, e.g. the use of father in \[19\] 
(synonymous with he, while in \[20\] the pronoun and in \[27\] the noun 
is necessary). 
3. To conclude, it should be noted that when speaking about the 
hierarchy of salience (activation) and of degrees of activation of 
items of the stock of shared knowledge we do not have in mind an 
absolute scale or linear ordering of elements. The raise or drop of 
activation should be always taken as related to the other items of 
the stock. In the rules, we use two kinds of raise: a mild one <Rule 
<3)), and a strong one <Rule (2)); it may be the case that further 
empirical research will lead us to a more subtle classification. 
Also, we are aware that this is a point where linguistic and psychol- 
ogical investigations should go hand in hand. Anyhow, we are cnnvinc- 
ed that such an empirical research, if guided by an attempt at a more 
explicit and systematic description, is of primary importance for 
(automatic) language comprehension, and as such also for all applicat- 
ional projects of text understanding. 
FOOTNOTES: 
1 See Sgall (1967b; 1975), Sgall and Haji~ov~ (1977), Haji~ov~ 
and Sgall (1980), Sgall, Haji~ov~ and Bene~ov~ (1973). 
2 The term "giveness" is often used in a similar sense, but it 
should be born in mind that a "given" item may be in contrast, or ir- 
recoverable (in the terms of Halliday), or not identifiable (Chafe), 
so that a definite noun may well occur inside the focus of a sentence. 
ON THE ROI.~ OF HIERARCHY OF ACTIVATION 113 
3 The TR of a sentence has the form of a dependency tree, 
labelled by complex symbols with the main predicate as its root; an 
ordering of the nodes is defined with respect to their so-called com- 
municative dynamism, fulfilling the condition of projectivity; for a 
detailed description of TR's in the framework of the functional gen- 
erative description, to which we subscribe, see Sgall (1967a); Haji- 
~ov~ and Sgall (1980); and Panevov~ in this volume. 

REFERENCES

\[i\] Chafe, W.L., Language and consciousness, Language 50(1974)111-133. 

\[2\] Dane~, F., Typy tematick~ch posloupnost/ v textu \[Types of them- 
atic progressions in text\], Slovo a slovesnost 29 (1968) 125-141. 

\[3\] Dane~, F., Zur semantischen und thematischen Struktur des Komuni- 
kats, in: Probleme der Textgrammatik (Berlin, 1976, 28-40). 

\[4\] Grosz, B.J., The representation and use of focus in dialogue 
understanding, Techn. Note 151, SRI Project 5844, 1977. 

\[5\] Haji~ov~, E. and Sgall, P., Dependency-based specification of 
topic and focus, SMIL (1980), Nr. 1-2, 93-140. 

\[6\] Haji~ov~,E. and Vrbov~, J., On the salience of the elements of 
the stock of shared knowledge, Folia linguistica (1982~. 

\[7\] Hirst, G., Anaphora in natural language understanding: a survey. 
(Springer Verlag, Berlin-Heidelberg-New York, 1981). 

\[8\] Hobbs, J.R., Pronoun resolution. Rep. 76-1, Dept. of Computer 
Science, City College, City Univ. of New York (1976). 

\[9\] Hobbs, J.R., Coherence and Coreference, Cognitive Science 3 
(1979) 67-90. 

\[iO\] McKeown, K., Paraphrasing using given and new information in 
question answer, unpublished masters thesis (Philadelphia 1979). 

\[ii\] Reichman, R., Conversational coherency, Rep. TR-17-78, Harvard 
Univ. (Cambridge, Mass., 1978). 

\[12\] Reichman, R.~ ~lain speaking: a theory and grammar of spontaneous 
discourse, R&p. 4681, Bolt Beranek and Newman Inc. (Cambridge, 
Mass., 1981). 

\[13\] Sgall, P., Generativni pop~s p~irozen4ho jazyka a ~esk~ deklinace 
\[Generative description of language and Czech declension\](Acade- 
mia, Prague 1967a). 

\[14\] Sgall, P., Functional sentence perspective in a generative de- 
scription, in: Prague Studies in Math.Ling. 2 (Prague, 1967b). 

\[15\] Sgall, P., Towards a pragmatically based theory of meaning, in: 
Searle, J.R., Kiefer, F. and Bierwisch,M. (eds.), Speech act 
theory and pragmatics (D.Reidel, Dordrecht, Holland, 198Oa). 

\[16\] Sgall, P., Text a reference \[Text and reference\], Slovo a sloves- 
nost 41 (198Ob~ 140-/45. 

\[17\] Sgall; P. and Haji~ov~, E., Focus on focus, Prague Bull. of Math. 
Linguistics 28 (1977) 5-51; 29 (1978) 22-41. 

\[18\] Sgall, P., Haji~ov~, E. and Bene~ov~, E., Topic, focus and gener- 
ative semantics (Skriptor, Kronberg/Ts., 1973). 

\[19\] Sgall, P., Nebesk~, L., Goral~/kov~, A. and Haji~ov~, E., A funct- 
ional approach to syntax in the generative description to language 
(American Elsevier, 1969). 
