Generating Referring Expressions:
Boolean Extensions of the Incremental
Algorithm
Kees van Deemter
 
University of Brighton
This paper brings a logical perspective to the generation of referring expressions, addressing
the incompleteness of existing algorithms in this area. After studying references to individual
objects, we discuss references to sets, including Boolean descriptions that make use of negated
and disjoined properties. To guarantee that a distinguishing description is generated whenever
such descriptions exist, the paper proposes generalizations and extensions of the Incremental
Algorithm of Dale and Reiter (1995).
1. Introduction
Generation of referring expressions (GRE) is a key task of most natural language gener-
ation (NLG) systems (e.g., Reiter and Dale 2000, Section 5.4). Regardless of the type of
knowledge base (KB) forming the input to the generator, many objects will not be des-
ignated in it via an ordinary proper name. A person like Mr. Jones, for example, may
be designated using an artificial name like #Jones083, if the name Jones is not uniquely
distinguishing. The same is true for a piece of furniture, a tree, or an atomic particle,
for instance, for which no proper name is in common use at all, or (in most cases) if
the generator tries to refer to an entire set of objects. In all such cases, the generator
has to “invent” a description that enables the hearer to identify the intended referent.
In the case of Mr. Jones, for example, the program could identify him by providing his
full name and address; in the case of a tree, some longer description may be necessary.
Henceforth, we will call the intended referent the target of the GRE algorithm.
The question that we set out to answer is whether existing GRE algorithms pro-
duce adequate descriptions whenever such descriptions exist: in short, whether these
algorithms are, as we shall say, complete. The paper brings a degree of formal preci-
sion to this issue and reveals a number of reasons why current GRE algorithms are
incomplete; we sketch remedies and discuss their consequences in terms of linguistic
coverage and computational tractability. We take the Incremental Algorithm (Dale and
Reiter 1995) to represent the state of the art in this area, and we minimize the devi-
ations from this algorithm. As a result, this paper might be read as an investigation
into how widely the ideas underlying the Incremental Algorithm can be used, and
the extent to which they may be generalized. The main generalization that we will
investigate involves complex Boolean combinations of properties, that is, descriptions
that involve more than a merely intersective (i.e., logically conjunctive) combination
of properties. Such generalizations are natural because the properties involved are im-
plicitly present in the KB, as we will explain; they become especially relevant when the
 Information Technology Research Institute (ITRI), University of Brighton, Lewes Road, Brighton BN2
4GJ, UK. E-mail: Kees.van.Deemter@itri.brighton.ac.uk.
c© 2002 Association for Computational Linguistics
Computational Linguistics Volume 28, Number 1
algorithms are also generalized to generate references to sets, rather than individual
objects. But, before we arrive at these generalizations, we will identify and confront a
number of cases in which current GRE algorithms are incomplete even with respect
to merely intersective descriptions.
In this paper, we will deal with “first mention” descriptions only (unlike Dale
1992, Chapter 5; Mittal et al. 1998; Kibble 1999), assuming that the information used
for generating the description is limited to a KB containing complete information about
which properties are true of each object. Also, we focus on “one shot” descriptions,
disregarding cases where an object is described through its relations with other ob-
jects (Dale and Haddock 1991; Horacek 1997; Krahmer, van Erk, and Verleg 2001).
More crucially, we follow Dale and Reiter (1995) in focusing on the semantic content
of a description (i.e., the problem of content determination, for short), assuming that
any combination of properties can be expressed by the NLG module responsible for
linguistic realization. This modular approach allows us to separate logical aspects of
generation (which are largely language independent) from purely linguistic aspects,
and it allows the realization module to base its decisions on complete information
about which combination of properties is to be realized. Accordingly, when we write
Generation of Referring Expressions or GRE, we will refer specifically to determination of
the semantic content of a description. Analogously, the word description will refer to
the semantic content of a linguistic expression only. Note that our modular approach
makes it unnatural to assume that a description is always expressed by a single noun
phrase: if several sentences are needed, then so be it.
After summarizing the Incremental Algorithm in Section 2, in Section 3 we take a
closer look at the algorithm in its standard, “intersective” form, in which it identifies
an object by intersecting a number of atomic properties. We discuss cases in which
this algorithm fails to find an adequate description even though such a description
exists, and we propose a number of possible remedies. Having extablished a com-
pleteness result for a version of the intersective Incremental Algorithm, we turn to
questions of completeness that involve more complex Boolean combinations in Sec-
tion 4. In Section 5, we summarize the main results of our exploration and put them
in perspective.
2. Dale and Reiter (1995): The Incremental Algorithm
The Incremental Algorithm of Dale and Reiter (1995) singles out a target object from
among some larger domain of entities. It does this by logically conjoining a num-
ber of properties found in a part of the KB that represents information shared be-
tween speaker and hearer. The authors observed that the problem of finding a (“Full
Brevity”) description that contains the minimum number of properties is computation-
ally intractable (i.e., NP Hard). They combined this with the known fact that speakers
often produce nonminimal descriptions anyway (e.g., Pechman 1989). Accordingly,
they proposed an algorithm that only approximates Full Brevity, while being of only
linear complexity. Our summary of the algorithm glosses over many details, yet still
allows us to discuss completeness. In particular, we disregard any special provisions
that might be made for the selection of head nouns because, arguably, this has to
involve realizational issues.
1
1 Compare Dale and Reiter (1995), where head nouns are taken into account during content
determination. Head nouns can also be selected during linguistic realization or by interleaving of
content determination and realization (e.g., Horacek 1997; Stone and Webber 1998; Krahmer and
Theune 1999).
38
van Deemter Generating Referring Expressions
The Incremental Algorithm produces a set L of properties P
1
,:::, P
n
such that their
logical conjunction forms a “distinguishing description” (Dale 1989) of the target object
r. In other words, writing [[Q]] for the extension of Q (i.e., the set of objects that have
the property Q), the intersection [[P
1
]]\   \[[P
n
]] must equal the singleton set frg.
It is a “hillclimbing” algorithm, which finds better and better approximations of the
target setfrgby accumulating more and more properties—hence the term Incremental.
There is no backtracking. Consequently, if some property P
i
in L is made redundant
by later additions (i.e., when ([[P
1
]]\   \[[P
i
−1]]\[[P
i
+ 1]]\   \[[P
n
]])  [[P
i
]]),
then P
i
is retained as a member of L nevertheless.
In the full algorithm (see below, D&R
Att
), properties are analyzed as pairs consist-
ing of an Attribute and a Value. Attributes are ordered in a listA.IfA
i
precedes A
j
in
A, then A
i
is “more preferred than” A
j
; as a consequence, A
i
will be considered before
A
j
by the algorithm. Suppose r is the target object, and D (the “domain”) is the set
of elements from which r is to be selected. The algorithm iterates through A; for each
Attribute A
i
, it checks whether specifying a Value for that Attribute would rule out
at least one object that has not already been ruled out; if so, the Attribute is added
to L, with a suitable Value (FindBestValue, below). C is the set of “confusables” at
any given stage of the algorithm.
2
Objects that are ruled out are removed from C. The
process of expanding L and contracting C continues until C = frg; if and when this
condition is met, L is a distinguishing set of properties.
For easy generalizability, the algorithm will be cast in set-theoretic terms. We first
present a version that focuses on properties, without separating these into Attributes
and Values, and assume the properties themselves are ordered in a list P (cf. Reiter
and Dale 2000). This version of the algorithm will be called D&R
Prop
,orD&R when
there is no risk of confusion. We assume that the domain contains one or more objects
other than the target object, the so-called distractors: thus, r2Dbut frg6=D.
L := fL is initialized to the empty setg
C :=DfC is initialized to the domaing
For each P
i
2Pdo
If r2[[P
i
]] & C6 [[P
i
]] fP
i
removes distractors from Cg then do
L := L[fP
i
gfProperty P
i
is added to Lg
C := C\[[P
i
]] fAll elements outside [[P
i
]] are removedg
If C =frg then Return L fSuccessg
Return FailurefAll properties in P have been tested, and still C6=frgg
Assuming (as do Dale and Reiter [1995]) that the tests in the body of the loop take
some constant amount of time, the worst-case running time is on the order of n
a
(i.e.,
O(n
a
)), where n
a
is the total number of properties. So, the algorithm has only linear
complexity.
A slightly closer approximation of Full Brevity can be achieved if Attributes and
Values are separated (Dale and Reiter 1995), allowing the algorithm to choose the
“best” Value for each Attribute. Given an Attribute, FindBestValue selects the Value
that removes most distractors while still including the target r. If no Value includes r,
the function returns nil. In case of a tie (i.e., no Value removes more distractors than
all others), FindBestValue chooses the least specific of the contestants. For example,
2 Thus, C contains r, unlike in Dale and Reiter (1995). The difference is purely presentational.
39
Computational Linguistics Volume 28, Number 1
when dog rules out as many distractors as chihuahua, chihuahua cannot be chosen. A
is the list of Attributes; L is the set of Attribute/Value combinations returned by the
algorithm. A further notational convention will be useful: Values will be identified
by two indices, the first of which identifies the Attribute. Thus, to denote Value j of
Attribute A
i
, we write V
i,j
. This version of the algorithm will be called D&R
Att
. The
initializations of L and D are omitted for brevity.
For each A
i
2Ado
V
i,j
= FindBestValue(r, A
i
)
If r2[[V
i,j
]]& C6 [[V
i,j
]] then do
L := L[fV
i,j
g
C := C\[[V
i,j
]]
If C =frg then Return L
Return Failure
We will switch back and forth between D&R and D&R
Att
, depending on what is
at stake. Like D&R, D&R
Att
has linear complexity. This can be made precise in the
following way.
3
If the running time of a call of FindBestValue(r, A
i
) is a constant
times the number of Values of the Attribute A
i
, then the worst-case running time of
D&R
Att
is O(n
v
n
a
), where n
a
equals the number of Attributes in the language and n
v
the average number of Values of all Attributes.
3. Completeness of the Incremental Algorithm
Some new definitions will be useful. A GRE algorithm is successful with respect to a
given situation (i.e., with respect to a KB and a target) if it produces a distinguishing
description of r in that situation. We will call an algorithm complete if it is successful
in every situation in which a distinguishing description exists. Success is not always
possible: the properties in the KB may not be sufficient for individuating a given object.
Such no-win situations will not be held against an algorithm.
The Incremental Algorithm generates descriptions that contain set intersection as
their only Boolean operation. We define a GRE algorithm to be intersectively complete
if it has the following property: whenever an object can be characterized by intersecting
a finite number of properties, the algorithm will find such an intersection. We would
like to prove the Incremental Algorithm to be intersectively complete, but we will
meet a few obstacles before we get there.
3.1 Completeness and Overlapping Values
One assumption without which the Incremental Algorithm cannot be proven to be
intersectively complete concerns the semantic relation between different Values of a
given Attribute: their extensions should not “overlap” in the following precise sense:
Values V
i,j
and V
i,k
(and equally, their extensions) overlap iff
V
i,j
\V
i,k
, V
i,j
−V
i,k
, and V
i,k
−V
i,j
are all nonempty.
3 Dale and Reiter arrived at linearity via the difficult concept of typical running time. They assumed that,
typically, n
l
(i.e., the number of properties in the description) is proportional to the number of
Attributes examined by the algorithm (Ehud Reiter, personal communication). This allowed them to
argue that the typical running time is O(n
d
n
l
), where n
d
is the number of distractors (Dale and Reiter
1995, Section 3.1). Our own worst-case assessment does not rely on assumptions of typicality.
40
van Deemter Generating Referring Expressions
(If V
i,j
and V
i,k
do not overlap, then either [[V
i,j
]] [[V
i,k
]],or[[V
i,k
]] [[V
i,j
]],or[[V
i,j
]]
and [[V
i,k
]] have an empty intersection.) Values can overlap for different reasons. Some
Attributes (e.g., COLOR) have “vague” Values (e.g., RED, ORANGE), which may be mod-
eled as overlapping: some objects may count as both red and orange. Also, Values may
derive from particular parts or aspects of an object; for example, if an object counts as
METAL (PLASTIC) because it has some METAL (PLASTIC) parts, then it may be listed as both
METAL and PLASTIC. Further examples arise if the KB models relations through unana-
lyzed properties. For example, a desk, or a particular type of desk, can stand in a given
relation (e.g., “being considered by” or “being bought by”) to more than one other
company. To see the problems arising from overlapping Values, consider a KB that
models which customer bought which types of desks, and where C =fa, b, c, d, e, fg:
BOUGHT-BY: PHILIPS (fa, b, eg), SONY (fa, c, d, fg)
COLOR: BROWN (fa, bg), YELLOW (fc, dg)
(Desks of types a, b, and e were bought by Philips, and so on. Note that desks of
type a were bought by two different companies.) Suppose a is the target, while the
Attribute BOUGHT-BY is more “preferred” than COLOR. The Value PHILIPS (being the
BestValue of BOUGHT-BY, since it removes more distractors than the Value SONY)is
chosen first, reducing the initial set C tofa, b, eg. Now, the algorithm is doomed to end
in Failure, since the different Values of COLOR are unable to remove the unwanted b
without also sacrificing a. None of this can be corrected, since the algorithm does not
use backtracking. Note that a uniquely identifying description of a would have been
possible if only SONY had been chosen instead of PHILIPS, leading to a description like
the brown desk bought by Sony. The algorithm does not just fail: it fails in a situation
where Success was perfectly achievable!
How can this limitation be remedied? One might introduce a limited kind of back-
tracking, which “remembers” where the algorithm has encountered overlapping Val-
ues and, when it results in Failure, goes back to the last-encountered situation where
it has made a choice between overlapping Values; if this does not lead to Success,
the algorithm backtracks to the previous choice situation, and so on until no more
choice situations are left (Failure) or a distinguishing description has been reached
(Success). Unfortunately, this algorithm becomes intractable if Values overlap too of-
ten: in the worst case, we are back to having to check all combinations of properties.
A simpler and computationally more efficient algorithm would include all over-
lapping Values that are true of the target while also removing some distractors. This
could be done as follows: whenever a Value V
i,j
of an Attribute A
i
is selected for in-
clusion in L, search for other Values of the same Attribute that have the target r as an
element; if such a Value V
i,k
is found, check whether it stands in the subset relation to
V
i,j
(i.e., either [[V
i,j
]] [[V
i,k
]] or [[V
i,k
]] [[V
i,j
]]); if not, then include V
i,k
as well; next,
search for yet another Value V
i,l
of the same Attribute that has r as an element, and
include V
i,l
if it does not stand in the subset relation to V
i,j
or V
i,k
; and so on until no
other Values of A
i
exist that have r as an element; then move on to the next Attribute.
This algorithm has a worst-case running time of O(n
a
n
2
v
).
4
In our example, this algorithm would produce a set consisting of the properties
BOUGHT BY SONY and BOUGHT BY PHILIPS, which can be realized as the desk bought by
Sony and by Philips; if we change the example by letting Philips buy c as well as a, the
4 This assumes that, once FindBestValue has found a Value V that removes distractors, one may need to
inspect all the other Values of the same Attribute to find Values overlapping with V. Shortcuts are
possible if Values are stored using a structure that reflects their semantic relationships.
41
Computational Linguistics Volume 28, Number 1
algorithm will go on to select the property BROWN, resulting in a set of properties that
may be realized as the brown desk bought by Sony and by Philips. Such descriptions appear
to be quite natural. One might even argue, on Gricean grounds (Grice 1975), that
identifying a simply as being bought by Philips can give rise to the false implicature
that a was not bought by Sony. This suggests that the proposed algorithm might also
be empirically more accurate than the one using limited backtracking provided, of
course, properties are properly aggregated (e.g., Dalianis and Hovy 1996).
3.2 Assumptions Concerning Infinite Sets
To prove intersective completeness, certain assumptions concerning the cardinality of
sets need to be made. To give an extreme example, suppose one wanted to refer to
a real number that does not have a “proper name” (unlike, e.g.,  ); then the class of
potentially useful properties is so vast that no GRE algorithm can take them all into
consideration. As long as the number of properties (i.e., Attribute/Value combina-
tions) is denumerably infinite (Kleene 1971), only termination becomes problematic: if a
uniquely referring description [[P
1
]]\   \[[P
n
]] exists, then the algorithm will find one
in finite time, since each of the n properties in the description will be found in finite
time; if no distinguishing description exists, however, the algorithm never terminates.
In the less likely case where the set of properties is nondenumerably infinite (i.e., it
does not stand in a 1-1 relation to any set of natural numbers), completeness becomes
problematic as well, since it is impossible for the algorithm to consider all properties;
hence, successful combinations may be overlooked (cf. Kleene 1971,pages 6–8).
Infinity of the set of distractors results in a different problem. The key question
is whether there exists an effective procedure for removing distractors (i.e., for cal-
culating C\[[P
i
]]). If no such procedure exists, the Incremental Algorithm can only
be applied after a property has been found that cuts down the set of distractors to a
manageable size. To be on the safe side when we prove completeness, we will assume
that the set of properties is at most denumerably infinite, while the set of distractors
is finite. These assumptions are harmless in connection with present NLG systems, all
of which work with relatively small sets. It is unclear how human speakers cope with
large sets of properties and/or distractors, but this question goes beyond our present
concerns.
3.3 Proving Intersective Completeness
Based on these considerations, we prove intersective completeness under some as-
sumptions concerning infinity and overlapping Values. We deal first with D&R, then
with the more complex D&R
Att
.
Theorem 1: Completeness of D&R Suppose there are at most denumerably many
properties, and finitely many (one or more) distractors. Then if an object can be individ-
uated by intersecting a finite number of properties, D&R will find such an intersection.
Proof Suppose [[Q
1
]] \   \[[Q
m
]] =frg, where the properties Q
1
,:::, Q
m
occur inP
in the order indicated by the subscripts. Now either D&R returns Success before it has
inspected all of Q
1
,:::, Q
m
, or it reaches the point where all of Q
1
,:::, Q
m
have been
inspected. This does not mean that all of Q
1
,:::, Q
m
have necessarily been included in
L, since other properties inP may have been selected that cause some of Q
1
,:::, Q
m
not
to remove any distractors. Yet, when all of Q
1
,:::, Q
m
have been inspected, Success
must have been achieved. To see this, let Des
i
be the description that results after
processing (i.e., inspecting and possibly including) Q
i
. Then a proof by induction over
42
van Deemter Generating Referring Expressions
i shows that [[Des
i
]]  [[Q
1
]] \   \[[Q
i
]], for all i  m. (Consider the basic case,
where i = 1, and assume that Q
1
62[[Des
1
]]. Q
1
was rejected, so it did not remove any
distractors; hence, [[Des
1
]]  [[Q
1
]]. The induction step is analogous.) It follows that
[[Des
m
]] [[Q
1
]]\   \[[Q
m
]] =frg. But r2[[Des
m
]],so[[Des
m
]] =frg.
Theorem 2: Completeness of D&R
Att
Assume (1) Attributes have no overlapping
Values (see Section 3.1), and (2) there are at most denumerably many Attributes and
Values, and finitely many (one or more) distractors. Then if an object can be individu-
ated by intersecting a finite number of properties, D&R
Att
will find such an intersection.
Proof Given Assumption (1), if D&R is complete, then so is D&R
Att
. To see this, let
BV abbreviate FindBestValue(r, A
i
). Suppose there is a Value V
i,j
of Attribute A
i
that
leads to a distinguishing description whereas BV does not. Then a contradiction is
derived as follows. For certain V
i,a1
,:::, V
i,an
,
[[V
i,a1
]]\   \[[V
i,an
]]\[[V
i,j
]] =frg, whereas
[[V
i,a1
]]\   \[[V
i,an
]]\[[BV ]]6=frg.
So, either (i) r62BV or (ii) there exists x6= r for which x2[[V
i,a1
]]\   \[[V
i,an
]]\BV .
But Case (i) contradicts the definition of FindBestValue (see Section 2); Case (ii), on
the other hand, implies that
x2[[V
i,a1
]]\   \[[V
i,an
]]\[[BV ]], whereas
x62[[V
i,a1
]]\   \[[V
i,an
]]\[[V
i,j
]];
hence, x 2 [[BV ]] while x 62 [[V
i,j
]]. But r 2 [[BV ]] \ [[V
i,j
]],so[[BV ]] and [[V
i,j
]] are
not disjoint. Consequently, by Assumption (1), Case (ii) implies that [[V
i,a1
]] \   \
[[V
i,an
]] \ [[V
i,j
]] is a real subset of [[V
i,a1
]] \   \[[V
i,an
]] \[[BV ]], contradicting the fact
that FindBestValue prefers a more general Value (i.e., BV ) over a more specific one
(i.e., V
i,j
) only if it removes the same distractors.
4. Generalizing the Incremental Algorithm
Both versions of the Incremental Algorithm have been proven to be intersectively com-
plete. Now we widen the issue to include all other Boolean combinations, involving
negation (i.e., complementation) and disjunction (i.e., union).
5
This is natural, since
properties expressed by Boolean combinations are implicit in the KB: if the KB lists the
property POODLE and the property ALSATIAN, then it implicitly contains the property
of being either a poodle or an Alsatian. This move will, however, only have its full
impact when we also widen the issue to reference to sets of objects.
In the new setting, it will be useful to generalize our earlier notion of intersec-
tive completeness (Section 3), calling a GRE algorithm Boolean complete iff it finds a
Boolean description of a set whenever one can be given on the basis of the properties
in the KB.
5 The well-known correspondence between set-theoretical operations like set union and propositional
operators like disjunction will allow us to “mix and match” the two terminologies.
43
Computational Linguistics Volume 28, Number 1
4.1 Describing Sets
Generating descriptions is even more important if the target is a set than if it is a single
object: even if the objects in the set have proper names, the set as a whole may lack
a name (and enumerating the objects may be cumbersome). Yet, reference to sets has
long been disregarded in NLG. In this section, we sketch generalizations of D&R that
produce descriptions of sets. To begin with, the algorithm D&R
Plural
finds intersections
P
1
\   \P
n
of atomic properties P
1
,:::, P
n
whose extension equals a given target set S
(van Deemter 2000). Since S may or may not be a singleton, D&R
Plural
subsumes D&R.
As before, we assume a nonempty set of distractors; that is, S Dbut S6=D.
6
L := 
C :=D
For each P
i
2Pdo
If S [[P
i
]] & C6 [[P
i
]] then do
L := L[fP
i
g
C := C\[[P
i
]]
If C = S then Return L fSuccessg
Return FailurefAll properties in P have been tested, yet C6= Sg
Note that S takes the place of the target object r in the earlier algorithms; the process
of expanding L and contracting C continues until C = S. Because this is basically the
same algorithm as D&R, it has the same computational complexity of O(n
a
), where n
a
is the cardinality of P.
D&R
Plural
characterizes a set by scrutinizing its elements. This does not work for
properties like BEING OF THE SAME AGE, which crucially pertain to sets of objects (cf.
Stone 2000). The algorithm can, however, be generalized to cover such cases if we
initialize C not toDbut to the powerset ofD, after which the algorithm selects properties
of sets, removing from P(D) all those sets for which the property is false. For example,
selection of BEING OF THE SAME AGE removes all those sets whose elements are not of
the same age as each other, selection of FORMING A FOOTBALL TEAM removes all sets
that do not make up a football team, and so on. As a result, the algorithm generates
descriptions of sets of collective entities (i.e., sets of sets). In this way, descriptions such
as those teams all of whose members are of the same age can be generated. In this collective
version of D&R
Plural
, the target S is a set of sets; P is a list of properties of sets, so
if P
i
2P, then [[P
i
]] is also a set of sets. As in the case of distributive properties,
describing one entity (in this case, one set) is a special case of describing a set of
entities.
L := 
C := P(D)
For each P
i
2Pdo
If S [[P
i
]] & C6 [[P
i
]] then do
L := L[fP
i
g
6 We continue to disregard special provisions for head nouns (see footnote 1). Note, however, that a
head noun must be selected that suits every element of the target set.
44
van Deemter Generating Referring Expressions
C := C\[[P
i
]]
If C = S then Return L fSuccessg
Return Failure
Once again, these adaptations leave the algorithm structurally unchanged: sets replace
objects throughout. Yet, they cause the complexity of the algorithm to become expo-
nential, since testing whether C = S involves inspecting all elements of C, of which
there can be up to 2
n
d
(where n
d
is the cardinality of the domain D).
This algorithm can also be applied to distributive properties if these are upgraded
to the level of sets: Let a newfangled distributive property be true of a set iff the
property (in ordinary parlance) is true of all its elements (Kamp and Reyle 1993,
page 338). This requires that the target S is always cast as a set of sets, even if it
is viewed distributively. For example, if a set of players—say, a, b, and c—are to
be characterized as a collection (e.g., to say that they won as a team of three), then
S = ffa, b, cgg; if they are to be characterized distributively (e.g., to say that each of
them has the flu), then S = ffa, b, cg,fa, bg,fa, cg,fb, cg,fag,fbg,fcgg. In this way, the
algorithm is able to combine collective and distributive properties, as in those football
teams whose members are British.
We will not explore collective versions of the Incremental Algorithm further here,
focusing instead on the relatively simple case of D&R
Plural
, in which all properties
are distributive. As in the case of D&R, it is easy to separate Attributes and Values
when referring to sets, allowing a closer approximation of Full Brevity: the resulting
algorithm, D&R
Plural,Att
,istoD&R
Plural
as D&R
Att
is to D&R; overlapping Values can be
treated as described in Section 3.1. In what follows, we will once again take property-
oriented versions of the Incremental Algorithm as our starting point, but implications
for the separation between Attributes and Values will be mentioned where they are
nontrivial.
4.2 Using Negations and Disjunctions
Now that we are able to generate references to sets, let us move away from purely
intersective descriptions, on to full Boolean combinations of properties. Consider a KB
whose domain is a set of animals (a, b, c, d, e) and whose only Attributes are TYPE and
COLOR:
TYPE: DOG (fa, b, c, d, eg), POODLE (fa, bg)
COLOR: BLACK (fa, b, cg), WHITE (fd, eg)
(All domain elements happen to be dogs.) In this situation, the Incremental Algorithm
does not allow us to individuate any of the animals. Intuitively, however, the KB
should enable one to refer to c, for example, since it is the only black dog that is not
a poodle:
fcg= BLACK\POODLE
A similar gap exists where disjunctions might be used. For example, the Incremental
Algorithm does not make the set of dogs that are either white or poodles referrable,
whereas it is referrable in English—for example, the white dogs and the poodles.
In the next two sections, we will investigate how negation and disjunction can
be taken into account in GRE. But first we introduce a trick for determining whether
45
Computational Linguistics Volume 28, Number 1
unique identification of an entity is possible, in a given situation.
7
The idea is to
calculate, for each element d in the domain, the Satellite set of d, that is, the intersection
of the extensions of all the properties true of d. Taking all extensions from our dog
example, we have
Satellites(a)=Satellites(b)=DOG\POODLE\BLACK =fa, bg
Satellites(c)=DOG\BLACK =fa, b, cg
Satellites(d)=Satellites(e)=DOG\WHITE =fd, eg
Satellite sets show which sets can be uniquely identified and which ones cannot. In the
case of the dogs, for example, no intersective description of fcg is possible because,
in the Satellite sets, c is always accompanied by other objects (i.e., a and b); more
generally, in this example, no object in the domain is uniquely identifiable, since no
object occurs in a Satellite set that is a singleton.
Satellite sets can also be applied to the construction of descriptions. The entityfa, bg,
for example, is uniquely described by the intersection DOG\POODLE\BLACK, and this
can be read off the list of Satellite sets. Two of the three properties in DOG\POODLE\
BLACK are redundant, however. Using Satellites sets for the construction of descriptions
can be particularly useful when properly generalized to Boolean descriptions, but
shortening the resulting descriptions in a computationally efficient way is difficult (van
Deemter and Halld ´orsson 2001). The present paper will focus on another approach to
Boolean descriptions, which takes the Incremental Algorithm as its point of departure
(van Deemter 2001).
4.3 Generating Boolean Descriptions
In this section, we will show how full Boolean descriptions can be generated. This
can be done in many different ways depending, among other things, on what form of
descriptions are preferred, for example, disjunctions of conjunctions, or conjunctions
of disjunctions. We will aim for the latter, while staying as close as possible to the
Incremental Algorithm. The algorithm proceeds as follows. First we add negations to
the list of atomic properties. Then D&R
Plural
runs a number of times: first, in Phase 1,
the algorithm is performed using all positive and negative literals; if this algorithm
ends before C = S, Phase 2 is entered in which further distractors are removed from
C by making use of negations of intersections of two literals, and so on, until either
C = S (Success) or all combinations have been tried (Failure). Observe that the
negation of an intersection comes down to set union, because of De Morgan’s Law:
P
1
\   \P
n
= P
1
[   [P
n
. Thus, Phase 2 of the algorithm deals with disjunctions of
length 2, Phase 3 deals with disjunctions of length 3, and so on. Optimizations may be
applied to shorten the resulting descriptions. For instance, a description of the form
(P[Q)\(P[R) can be simplified to (P[(Q\R)) using standard algorithms (e.g.,
McCluskey 1965). Such optimizations, however, are less urgent than in the case of
the more verbose descriptions generated using Satellite sets (see above), and we will
disregard optimizations here.
A schematic presentation may be useful, in which P
+=−
stands for any literal,
that is, any atomic property or its negation. (Different occurrences of P
+=−
denote
potentially different literals.) The length of a property will equal the number of literals
7 The idea of Satellite sets is due to Magn ´us Halld´orsson (personal communication).
46
van Deemter Generating Referring Expressions
occurring in it. We will say that a D&R
Plural
phase uses a set of properties X if it loops
through the properties in X (i.e., X takes the place of P in the original D&R
Plural
).
D&R
Boolean
Phase 1. Perform D&R
Plural
using all properties of the form P
+=−
. If this is
successful, then stop; otherwise, go to Phase 2.
Phase 2. Based on the Values of L and C coming out of Phase 1, perform
D&R
Plural
using all properties of the form P
+=−
[P
+=−
. If this is successful,
then stop; otherwise, go to Phase 3.
Phase 3. Based on the Values of L and C coming out of Phase 2, perform
D&R
Plural
using all properties of the form P
+=−
[P
+=−
[P
+=−
. If this is
successful, then stop; otherwise, go to Phase 4.
Etc.
One can require without loss of generality that no property, considered at any phase,
may have different occurrences of the same atom. (For example, it is useless to consider
the property P
1
[P
2
[P
1
, which must be true of any element in the domain, or the
property P
1
[P
2
[P
1
, which is equivalent to the earlier-considered property P
1
[P
2
.)
Therefore, since at phase n there is room for properties of length n, the maximal number
of phases equals the total number of atomic properties.
Consider our old example, where the preference order of atomic properties cor-
responds with the order in which they are listed, and where the same order extends
to their negations, all of which are less preferred. Abbreviating B = BLACK, D = DOG,
P = POODLE, and W = WHITE, we haveP =hB, D, P, W, B, D, P, Wi. Now, if S =fc, d, eg
and S = fcg (as before) are to be characterized, nothing eventful happens. In both
cases, a description is found during Phase 1: P in the first case, B\P in the second.
The situation gets more interesting if S = fa, b, d, eg, which triggers Phase 2. For in-
stance, if positive literals precede negative literals, the properties relevant for Phase 2
might be ordered as follows:
hB[D, B[P, B[W, D[P, D[W, P[W, B[D, B[P, B[W, D[B,
D[P, D[W, P[B, P[D, P[W, W[B, W[D, W[P, B[D, B[P,
B[W, D[P, D[W, P[Wi
During Phase 1, no property is selected, since the only property true of all elements in
S =fa, b, d, egis D, which fails to remove any distractors. During Phase 2, one property
after another is rejected. For example, the property B[D is rejected because it does not
remove any distractors. The first property that is true of all elements of S while also
removing distractors is P[W. This property removes all distractors at once, causing
the algorithm to end with L = fPOODLE[WHITEg as the complete description. If we
modify the example by letting [[BLACK]] =fa, cg(rather thanfa, b, cg) and S =fb, c, d, eg
(rather than S =fa, b, d, eg), then the description L =fBLACK[POODLEg is found.
D&R
Boolean
is incremental not only within a phase, but also from one phase to
the next, which causes shorter disjunctions to be favored over longer ones. Once a
property has been selected, it will not be abandoned even if properties selected during
later phases make it logically superfluous. As a result, one may generate descriptions
like X \ (Y [ Z) (e.g., white (cats and dogs)) in a situation where Y[Z (e.g., cats and
dogs) would have sufficed (because (Y [ Z)  X). This is not unlike some of the
redundancies generated by Dale and Reiter’s algorithm and, as in their case, it is
47
Computational Linguistics Volume 28, Number 1
unclear whether this is descriptively adequate. Adaptations can be made if needed.
For instance, phases might run separately before running in combination: first (as
usual) Phase 1, then 2, then (as usual) 1&2, then 3, then 1&3, then 2&3, then (as usual)
1&2&3, and so on.
8
As a result of this adaptation, the description Y[Z would be
generated because of Phase 2 alone.
Double incrementality, however, does not save D&R
Boolean
from intractability. To
estimate running time as a function of the number of properties (n
a
) in the KB and
those in the description (n
l
), we can mirror an argument in Dale and Reiter (1995,
Section 3.1.1) to show that the maximal number of properties to be considered equals
n
l
X
i=1
2
 
n
a
i
!
=
n
l
X
i=1
2
n
a
!
i!(n
a
−i)!
:
(The factor of 2 derives from inspecting both each atom and its negation.) If n
l
 n
a
,
then this is on the order of n
n
l
a
. To avoid intractability, the algorithm can be pruned.
No matter where this is done, the result is a polynomial algorithm. By cutting off
after Phase 1, for example, only (negations of) atomic properties are combined, pro-
ducing such descriptions as the black dog that is not a poodle, disregarding more complex
descriptions; as a result, completeness is lost, but only for references to nonsingle-
ton sets, because set union does not add descriptive power where the description of
singletons is concerned. The number of properties to be considered by this simpler
algorithm equals (n
a
)
2
+ 2n
a
−1. To produce descriptions like WHITE\(CAT[DOG) (i.e.,
white (cats and dogs)) as well, the algorithm can be cut off one phase later, leading to a
worst-case running time of O(n
3
a
), and so on for more and more complex descriptions.
D&R
Boolea
can, of course, be modified to take advantage of the distinction between
Attributes and Values. Suppose, for example, that V
1
[   [V
n
takes precedence over
W
1
[   [W
n
whenever there are more negative Values among V
1
,:::, V
n
than among
W
1
,:::, W
n
. Then the preference ordering between Attributes may be taken into ac-
count if the number of negative Values is the same in both unions; in case of a tie, the
number of distractors removed by each of the two unions may decide; if all this fails to
tip the balance, the relative specificity of Attributes may be used. The situation resem-
bles that of D&R
Att
but, in the case of the new algorithm, D&R
Boolean,Att
, there is more
scope for choice, because it compares combinations (i.e., unions) of properties: when the
preference order of individual Attributes has been decided, it can happen that [[V
i
]] is
more preferred than [[W
j
]], while [[W
k
]] is more preferred than [[V
l
]], in which case it is
unclear whether V
1
[   [V
n
should be more preferred or W
1
[   [W
n
. (Problems of
this kind are not specific to Boolean combinations. For example, if an object x is iden-
tified through the relation R(xy) and the predicate P(y), then the degrees of preference
of both R and P are relevant, and it is unclear which of the two is more important.)
Once D&R
Boolean,Att
is constructed along these lines, the question of overlapping
Values arises in exactly the same way as in the case of D&R
Att
and D&R
Plural,Att
. The
problem arises if components of different unions overlap, as when the algorithm com-
pares V
i,j
[V
k,l
and V
i,j
[V
k,l
0, where V
k,l
and V
k,l
0 overlap in the sense of Section 3.1: as in
the case of D&R
Att
, simply choosing the option that removes the most distractors may
cause the algorithm to become incomplete. This problem can be overcome as before, us-
ing either limited backtracking or inclusion of all relevant options (Section 3.1). Instead
of exploring D&R
Boolean,Att
any further, we will return to its predecessor, D&R
Boolean
,
to prove that it is powerful enough to do its job.
8 This possibility was suggested to me by Richard Power (personal communication).
48
van Deemter Generating Referring Expressions
4.4 Proving Boolean Completeness
In Section 3.3, we proved intersective completeness for two versions of Dale and Re-
iter’s Incremental Algorithm, D&R and D&R
Att
. We now prove Boolean completeness
for D&R
Boolean
, the Boolean extension of D&R
Plural
.
Theorem 3: Completeness of D&R
Boolean
Assume there are at most denumerably
many properties, and finitely many distractors (one or more). Then if a set can be
individuated distributively by any Boolean combination of properties, D&R
Boolean
will
find such a combination.
Proof Any Boolean expression can be written in conjunctive normal form (CNF), that
is, as an intersection of unions of literals (e.g., Fitting 1996). Theorem 3 follows from
the following lemma.
Lemma Let ’ be a CNF formula whose longest union has a length of n (i.e., it conjoins
n literals). Then D&R
Boolean
will find a description ’
0
that is coextensive with ’,inat
most n phases. This is proven by induction on the size of n.
Basic case: If n = 1, the lemma is equivalent to completeness of D&R
Plural
, the proof of
which is analogous to that of the completeness of D&R, replacing frg by S.
Induction step: Suppose the lemma is true for all n < i. Now consider a CNF ’ whose
longest union has length i; let ’ contain m unions of length i, namely, ’
1
\   \’
m
.
Then ’ can be written as the CNF  \’
1
\   \’
m
, where all the unions in  have
length < i. The lemma is true for all n < i,soif is sent to D&R
Boolean
, then the
output is some  
0
such that [[ 
0
]]=[[ ]], in fewer than i phases; so if, instead, ’ is
sent to D&R
Boolean
, then, after i−1 phases, some possibly incomplete description  
has been found, such that [[ ]] [[ ]]. Also, [[’]] [ ]]. Phase i inspects all unions of
length i, including each of ’
1
,:::,’
m
. Therefore, unless a description coextensive with
’ is found before phase i, one will be found during phase i. To see this, suppose the
algorithm finds  such that [[ ]]=[[’
1
]]\   \[[’
m
]]. Then [[ ]]\[[ ]]=[[’]]; but
[[’]] [ ]] [[ ]], therefore also [[ ]]\[[ ]]=[[’]].
5. Conclusion
The GRE algorithms discussed in this paper are fairly limited in their aspirations. For
example, they do not involve relational descriptions (Dale and Haddock 1991; Horacek
1997; Krahmer, van Erk, and Verleg 2001) or properties that are vague or context
dependent (van Deemter 2000). Moreover, they disregard shades of salience (unlike
algorithms proposed in Krahmer and Theune [1999], Theune [2000]), relying instead
on a simple dichotomy between those objects that are salient enough (which end up
in the domain D) and those that are not (Reiter and Dale 2000, Section 5.4). Finally,
like all other GRE algorithms that we are aware of, they disregard the generation of
descriptions in intensional contexts (e.g., John knows that x is the murderer of Jones; Dowty,
Wall, and Peters 1981).
But even within this limited brief, existing algorithms are incomplete. In particular,
we have shown Dale and Reiter’s (1995) Incremental Algorithm to be intersectively
incomplete with respect to Attributes that have overlapping Values and (less surpris-
49
Computational Linguistics Volume 28, Number 1
ingly) in some situations where the class of properties is infinitely large. Furthermore,
the Incremental Algorithm excludes reference to sets and limits itself to purely in-
tersective combinations of atomic properties, causing the algorithm to be incomplete
with respect to the set of all Boolean combinations. Having noted these shortcomings,
we have modified the Incremental Algorithm in such a way that these limitations are
removed. The result is a set of generalizations of the Incremental Algorithm, for which
we have proven completeness under appropriate assumptions.
Integration of these different algorithms into one unified algorithm would be a
nontrivial enterprise, as we have shown in Section 4.3. Integration with previously
proposed extensions of the Incremental Algorithm would raise further questions, stem-
ming from the fact that our descriptions are structurally complex. For example, con-
sider the treatment of relational properties. Which is better: adding a relational property
to a given incomplete description (:::in the wooden shed) or adding a negated property
(:::which is not a poodle)? Making informed decisions about such questions, with proper
attention to their combined effects, is a difficult task that is perhaps best tackled us-
ing the graph-theoretical approach outlined by Krahmer, van Erk, and Verleg (2001).
Their approach is specifically suitable for accommodating different GRE algorithms
and treats relations in the same way as properties
Brevity. We have assumed that, on the whole, descriptions ought to be as brief as
they can, as long as they are uniquely identifying. But in fact, a description can contain
much more than is logically necessary for identification, even beyond the redundan-
cies allowed by the Incremental Algorithm. Logically superfluous properties can, for
example, be motivated by “overloading” if they serve communicative purposes other
than identification (Pollack 1991; Dale and Reiter 1995, Section 2.4; Stone and Webber
1998; Jordan 1999). A description may also contain fewer properties than would be
necessary for identification—for example, when no distinguishing description exists.
A nondistinguishing description may take the form either of a definite description (as
in John’s son, when John has several sons) or of an indefinite description (as in one of
John’s sons; Horacek 1997; Stone and Webber 1998; Krahmer and Theune 1999). In both
cases, the description may be useful even though it fails to be distinguishing.
Tractability. Computational tractability has also been paramount in our explo-
rations. There is no agreement on the extent to which computational linguists should
worry about the computational complexity of algorithms, or about the precise way in
which complexity is most relevant (e.g., “typical” or worst-case complexity, cf. foot-
note 3). Far from aiming to speak the last word on these issues, the material discussed
here does shed some light on them. For example, even a fast algorithm can require
a large number of calculations, in which case a solution may never be found; in the
case of GRE, this happens when the set of distractors or the set of properties becomes
extremely large (Section 3.2). Conversely, a complex algorithm can be safe to use if the
domain is small (or if key calculations can be performed offline; e.g., Bateman 1999).
This may be achieved by putting a bound on the size of the search space, and this may
be justifiable on empirical grounds (see the discussion of D&R
Boolean
in Section 4.3).
One might, on the other hand, argue that bounding does not eliminate the disadvan-
tages of an otherwise intractable algorithm, because the true nature of an algorithm is
best revealed “by considering how it operates on unlimited cases” (Barton, Berwick,
and Ristad 1987, Section 1.4.1). Be this as it may, we believe that complexity theory
can offer valuable insights into the structure of GRE algorithms and that the grow-
ing attention to complexity in this area is a healthy development even if the practical
implications are not always straightforward.
Recent work also highlights an interesting mirror image of GRE complexity: a
logically superfluous property may make it easier for the reader to find the referent.
50
van Deemter Generating Referring Expressions
An interesting class of cases is explored in Paraboni (2000), which focuses on de-
scriptions of document parts. Consider the description the medicine depicted in Section
2.3. If Section 2 happens to contain only one figure, then the description the medicine
depicted in Section 2 would have been logically sufficient, but this description would
have made it necessary for the reader, in the worst case, to search through all of Sec-
tion 2, making it less useful. Examples of this kind suggest that GRE should also take
the computational complexity of interpretation into account. Experimental research on
“minimal cooperative effort” (Clark 1992; Cremers 1996) points in the same direc-
tion.
Acknowledgments
Thanks are due to Robert Dale, Magn ´us
Halld´orsson, Emiel Krahmer, Paul Piwek,
Richard Power, Ehud Reiter, and Matthew
Stone for useful discussions. Helpful
comments from the reviewers of
Computational Linguistics are also gratefully
acknowledged.

References
Barton, G. Edward, Robert C. Berwick, and
Eric Sven Ristad. 1987. Computational
Complexity and Natural Language. MIT
Press, Cambridge, MA.
Bateman, John A. 1999. Using aggregation
for selecting content when generating
referring expressions. In Proceedings of the
37th Annual Meeting of the Association for
Computational Linguistics, pages 127–134,
College Park, MD.
Clark, Herbert H. 1992. Arenas of Language
Use. CSLI Publications, Stanford, CA.
Cremers, Anita. 1996. Reference to Objects: An
Empirically Based Study of Task-Oriented
Dialogues. Ph.D. thesis, University of
Eindhoven.
Dale, Robert. 1989. Cooking up referring
expressions. In Proceedings of the 27th
Annual Meeting of Association for
Computational Linguistics, pages 68–75,
Vancouver.
Dale, Robert. 1992. Generating Referring
Expressions: Constructing Descriptions in a
Domain of Objects and Processes. MIT Press,
Cambridge, MA.
Dale, Robert and N. Haddock. 1991.
Generating referring expressions
involving relations. In Proceedings of the
9th Meeting of the European Chapter of the
Association for Computational Linguistics,
pages 161–166, Berlin.
Dale, Robert and Ehud Reiter. 1995.
Computational interpretations of the
Gricean maxims in the generation of
referring expressions. Cognitive Science, 18,
233–263.
Dalianis, Hercules and Eduard Hovy. 1996.
Aggregation in natural language
generation. In G. Adorni and M. Zock,
editors, Trends in Natural Language
Generation: An Artificial Intelligence
Perspective. Springer-Verlag, New York,
pages 88–105.
Dowty, David, Robert Wall, and Stanley
Peters. 1981. Introduction to Montague
Semantics. D. Reidel, Dordrecht.
Fitting, Melvin. 1996. First-Order Logic and
Automated Theorem Proving.
Springer-Verlag, New York.
Grice, Paul. 1975. Logic and conversation. In
P. Cole and J. Morgan, editors, Syntax and
Semantics: Vol. 3: Speech Acts. Academic
Press, New York, pages 43–58.
Horacek, Helmut. 1997. An algorithm for
generating referential descriptions with
flexible interfaces. In Proceedings of the 35th
Annual Meeting of the Association for
Computational Linguistics, pages 206–213.
Madrid.
Jordan, Pamela. 1999. Contextual influences
on the attributes included in repeated
descriptions. In Proceedings of the Workshop
“Generation of Nominal Expressions,” 11th
European Summer School in Logic, Language,
and Information (ESSLLI’99), Utrecht.
Augmented version to appear in K. van
Deemter and R. Kibble, editors,
Information Sharing. CSLI Publications,
Stanford, CA.
Kamp, Hans and Uwe Reyle. 1993. From
Discourse to Logic. Kluwer, Dordrecht.
Kibble, Rodger. 1999. Cb or not Cb?
Centering Theory applied to NLG.In
Proceedings of the ACL Workshop
“Discourse/Dialogue Structure and
Reference,” College Park, MD.
Kleene, Stephen C. 1971. Introduction to
Metamathematics. Wolters-Noordhoff.
Groningen.
Krahmer, Emiel and Mari¨et Theune. 1999.
Generating descriptions in context. In
Proceedings of the Workshop “Generation of
Nominal Expressions,” 11th European
Summer School in Logic, Language, and
Information (ESSLLI’99), Utrecht.
Augmented version to appear in K. van
Deemter and R. Kibble, editors,
Information Sharing. CSLI Publications,
Stanford, CA.
Krahmer, Emiel, Sebastiaan van Erk, and
Andr ´e Verleg. 2001. A meta-algorithm for
the generation of referring expressions. In
Proceedings of the 8th European Workshop on
Natural Language Generation (EWNLG’01),
pages 29–38, Toulouse, France.
McCluskey, Edward J. 1965. Introduction to
the Theory of Switching. McGraw-Hill, New
York.
Mittal, Vibhu O., Joanna D. Moore,
Giuseppe Carenini, and Steven Roth.
1998. Describing complex charts in
natural language: A caption generation
system. Computational Linguistics, 24,
431–468.
Paraboni, Ivandr ´e. 2000. An algorithm for
generating document-deictic references.
In Proceedings of the Workshop “Coherence in
Generated Multimedia,” 1st International
Conference on Natural Language Generation
(INLG’00), pages 27–31, Mitzpe Ramon,
Israel.
Pechman, Thomas. 1989. Incremental speech
production and referential overspecifi-
cation. Linguistics, 27, 89–110.
Pollack, Martha E. 1991. Overloading
intentions for efficient practical reasoning.
No ˆus, 25: 513–536.
Reiter, Ehud and Robert Dale. 2000. Building
Natural Language Generation Systems.
Cambridge University Press, Cambridge,
UK.
Stone, Matthew. 2000. On identifying sets. In
Proceedings of the 1st International Conference
on Natural Language Generation (INLG’00),
pages 116–123, Mitzpe Ramon, Israel.
Stone, Matthew and Bonnie Webber. 1998.
Textual economy through close coupling
of syntax and semantics. In Proceedings of
the 9th International Workshop on Natural
Language Generation (INLG’98),
pages 178–187, Niagara-on-the-Lake,
Canada.
Theune, Mari¨et. 2000. From Data to Speech:
Language Generation in Context. Ph.D.
thesis, University of Eindhoven.
van Deemter, Kees. 2000. Generating vague
descriptions. In Proceedings of the 1st
International Conference on Natural Language
Generation (INLG’00), pages 179–185,
Mitzpe Ramon, Israel.
van Deemter, Kees. 2001. Generating
referring expressions: Beyond the
incremental algorithm. In Proceedings of the
4th International Workshop on Computational
Semantics (IWCS’01), pages 50–66, Tilburg,
The Netherlands.
van Deemter, Kees and Magn ´us
Halld ´orsson. 2001. Logical form
equivalence: The case of referring
expressions generation. In Proceedings of
the 8th European Workshop on Natural
Language Generation (EWNLG’01),
pages 21–28, Toulouse, France.
