Corpus-Based Identification of Non-Anaphoric Noun Phrases 
David L. Bean and Ellen Riloff 
Department of Computer Science 
University of Utah 
Salt Lake City, Utah 84112 
{bean,riloff}@cs.utah.edu 
Abstract 
Coreference resolution involves finding antecedents 
for anaphoric discourse entities, such as definite 
noun phrases. But many definite noun phrases are 
not anaphoric because their meaning can be un- 
derstood from general world knowledge (e.g., "the 
White House" or "the news media"). We have 
developed a corpus-based algorithm for automat- 
ically identifying definite noun phrases that are 
non-anaphoric, which has the potential to improve 
the efficiency and accuracy of coreference resolu- 
tion systems. Our algorithm generates lists of non- 
anaphoric noun phrases and noun phrase patterns 
from a training corpus and uses them to recognize 
non-anaphoric noun phrases in new texts. Using 
1600 MUC-4 terrorism news articles as the training 
corpus, our approach achieved 78% recall and 87% 
precision at identifying such noun phrases in 50 test 
documents. 
1 Introduction 
Most automated approaches to coreference res- 
olution attempt to locate an antecedent for ev- 
ery potentially coreferent discourse entity (DE) 
in a text. The problem with this approach is 
that a large number of DE's may not have an- 
tecedents. While some discourse entities such 
as pronouns are almost always referential, def- 
inite descriptions I may not be. Earlier work 
found that nearly 50% of definite descriptions 
had no prior referents (Vieira and Poesio, 1997), 
and we found that number to be even higher, 
63%, in our corpus. Some non-anaphoric def- 
inite descriptions can be identified by looking 
for syntactic clues like attached prepositional 
phrases or restrictive relative clauses. But other 
definite descriptions are non-anaphoric because 
readers understand their meaning due to com- 
mon knowledge. For example, readers of this 
1In this work, we define a definite description to be a 
noun phrase beginning with the. 
paper will probably understand the real world 
referents of "the F.B.I.," "the White House," 
and "the Golden Gate Bridge." These are in- 
stances of definite descriptions that a corefer- 
ence resolver does not need to resolve because 
they each fully specify a cognitive representa- 
tion of the entity in the reader's mind. 
One way to address this problem is to cre- 
ate a list of all non-anaphoric NPs that could 
be used as a filter prior to coreference resolu- 
tion, but hand coding such a list is a daunt- 
ing and intractable task. We propose a corpus- 
based mechanism to identify non-anaphoric NPs 
automatically. We will refer to non-anaphoric 
definite noun phrases as existential NPs (Allen, 
1995). Our algorithm uses statistical methods 
to generate lists of existential noun phrases and 
noun phrase patterns from a training corpus. 
These lists are then used to recognize existen- 
tial NPs in new texts. 
2 Prior Research 
Computational coreference resolvers fall into 
two categories: systems that make no at- 
tempt to identify non-anaphoric discourse en- 
tities prior to coreference resolution, and those 
that apply a filter to discourse entities, identify- 
ing a subset of them that are anaphoric. Those 
that do not practice filtering include decision 
tree models (Aone and Bennett, 1996), (Mc- 
Carthy and Lehnert, 1995) that consider all pos- 
sible combinations of potential anaphora and 
referents. Exhaustively examining all possible 
combinations is expensive and, we believe, un- 
necessary. 
Of those systems that apply filtering prior to 
coreference resolution, the nature of the filter- 
ing varies. Some systems recognize when an 
anaphor and a candidate antecedent are incom- 
patible. In SRI's probabilistic model (Kehler, 
373 
The ARCE battalion command has reported that about 50 peasants of various ages have been 
kidnapped by terrorists of the Farabundo Marti National Liberation Front \[FMLN\] in San 
Miguel Department. According to that garrison, the mass kidnapping took place on 30 December 
in San Luis de la Reina. The source added that the terrorists forced the individuals, who were 
taken to an unknown location, out of their residences, presumably to incorporate them against their 
will into clandestine groups. 
Figure 1: Anaphoric and Non-Anaphoric NPs (definite descriptions highlighted.) 
1997), a pair of extracted templates may be 
removed from consideration because an out- 
side knowledge base indicates contradictory fea- 
tures. Other systems look for particular con- 
structions using certain trigger words. For ex- 
ample, pleonastic 2 pronouns are identified by 
looking for modal adjectives (e.g. "necessary") 
or cognitive verbs (e.g. "It is thought that...") 
in a set of patterned constructions (Lappin and 
Leass, 1994), (Kennedy and Boguraev, 1996). 
A more recent system (Vieira and Poesio, 
1997) recognizes a large percentage of non- 
anaphoric definite noun phrases (NPs) during 
the coreference resolution process through the 
use of syntactic cues and case-sensitive rules. 
These methods were successful in many in- 
stances, but they could not identify them all. 
The existential NPs that were missed were ex- 
istential to the reader, not because they were 
modified by particular syntactic constructions, 
but because they were part of the reader's gen- 
eral world knowledge. 
Definite noun phrases that do not need to be 
resolved because they are understood through 
world knowledge can represent a significant por- 
tion of the existential noun phrases in a text. In 
our research, we found that existential NPs ac- 
count for 63% of all definite NPs, and 24% of 
them could not be identified by syntactic or lex- 
ical mea.ns. This paper details our method for 
identifying existential NPs that are understood 
through general world knowledge. Our system 
requires no hand coded information and can rec- 
ognize a larger portion of existential NPs than 
Vieira and Poesio's system. 
3 Definite NP Taxonomy 
To better understand what makes an NP 
anaphoric or non-anaphoric, we found it useful 
to classify definite NPs into a taxonomy. We 
2Pronouns that are semantically empty, e.g. "It is 
clear that...." 
first classified definite NPs into two broad cat- 
egories, referential NPs, which have prior refer- 
ents in the texts, and existential NPs, which do 
not. In Figure 1, examples of referential NPs 
are "the mass kidnapping," "the terror- 
ists" and "the individuals.", while examples 
of existential NPs are "the ARCE battalion 
command" and "the Farabundo Marti Na- 
tional Liberation Front." (The full taxon- 
omy can be found in Figure 2.) 
We should clarify an important point. When 
we say that a definite NP is existential, we say 
this because it completely specifies a cognitive 
representation of the entity in the reader's mind. 
That is, suppose "the F.B.I." appears in both 
sentence 1 and sentence 7 of a text. Although 
there may be a cohesive relationship between 
the noun phrases, because they both completely 
specify independently, we consider them to be 
non-anaphoric. 
Definite Noun Phrases 
- Referential 
- Existential 
- Independent 
- Syntactic 
- Semantic 
- Associative 
Figure 2: Definite NP Taxonomy 
We further classified existential NPs into two 
categories, independent and associative, which 
are distinguished by their need for context. In- 
dependent existentials can be understood in iso- 
lation. Associative existentials are inherently 
associated with an event, action, object or other 
context 3. In a text about a basketball game, 
for example, we might find "the score," "the 
hoop" and "the bleachers." Although they may 
3Our taxonomy mimics Prince's (Prince, 1981) in 
that our independent existentials roughly equate to her 
new class, our associative existentials to her inferable 
class, and our referentials to her evoked class. 
374 
not have direct antecedents in the text, we 
understand what they mean because they are 
all associated with basketball games. In isola- 
tion, a reader would not necessarily understand 
the meaning of "the score" because context is 
needed to disambiguate the intended word sense 
and provide a complete specification. 
Because associative NPs represent less than 
10% of the existential NPs in our corpus, our ef- 
forts were directed at automatically identifying 
independent existentials. Understanding how 
to identify independent existential NPs requires 
that we have an understanding of why these 
NPs are existential. We classified independent 
existentials into two groups, semantic and syn- 
tactic. Semantically independent NPs are exis- 
tential because they are understood by readers 
who share a collective understanding of current 
events and world knowledge. For example, we 
understand the meaning of "the F.B.I." without 
needing any other information. Syntactically 
independent NPs, on the other hand, gain this 
quality because they are modified structurally. 
For example, in "the man who shot Liberty Va- 
lence," "the man" is existential because the rel- 
ative clause uniquely identifies its referent. 
4 Mining Existential NPs from a 
Corpus 
Our goal is to build a system that can identify 
independent existential noun phrases automati- 
cally. In the previous section, we observed that 
"existentialism" can be granted to a definite 
noun phrase either through syntax or seman- 
tics. In this section, we introduce four methods 
for recognizing both classes of existentials. 
4.1 Syntactic Heuristics 
We began by building a set of syntactic heuris- 
tics that look for the structural cues of restric- 
tive premodification and restrictive postmod- 
ification. Restrictive premodification is often 
found in noun phrases in which a proper noun 
is used as a modifier for a head noun, for ex- 
ample, "the U.S. president." "The president" 
itself is ambiguous, but "the U.S. president" is 
not. Restrictive postmodification is often rep- 
resented by restrictive relative clauses, preposi- 
tional phrases, and appositives. For example, 
"the president of the United States" and "the 
president who governs the U.S." are existen- 
tial due to a prepositional phrase and a relative 
clause, respectively. 
We also developed syntactic heuristics to rec- 
ognize referential NPs. Most NPs of the form 
"the <number> <noun>" (e.g., "the 12 men") 
have an antecedent, so we classified them as ref- 
erential. Also, if the head noun of the NP ap- 
peared earlier in the text, we classified the NP 
as referential. 
This method, then, consists of two groups of 
syntactic heuristics. The first group, which we 
refer to as the rule-in heuristics, contains seven 
heuristics that identify restrictive premodifica- 
tion or postmodification, thus targeting existen- 
tial NPs. The second group, referred to as the 
rule-out heuristics, contains two heuristics that 
identify referential NPs. 
4.2 Sentence One Extractions (Sl) 
Most referential NPs have antecedents that pre- 
cede them in the text. This observation is the 
basis of our first method for identifying seman- 
tically independent NPs. If a definite NP occurs 
in the first sentence 4 of a text, we assume the 
NP is existential. Using a training corpus, we 
create a list of presumably existential NPs by 
collecting the first sentence of every text and 
extracting all definite NPs that were not classi- 
fied by the syntactic heuristics. We call this list 
the S1 extractions. 
4.3 Existential Head Patterns (EHP) 
While examining the S1 extractions, we found 
many similar NPs, for example "the Salvadoran 
Government," "the Guatemalan Government," 
and "the U.S. Government." The similarities 
indicate that some head nouns, when premod- 
ified, represent existential entities. By using 
the S1 extractions as input to a pattern gen- 
eration algorithm, we built a set of Existen- 
tial Head Patterns (EHPs) that identify such 
constructions. These patterns are of the form 
"the <x+> 5 <nounl ...nounN>" such as "the 
<x+> government" or "the <x+> Salvadoran 
government." Figure 3 shows the algorithm for 
creating EHPs. 
4Many of the texts we used were newspaper arti- 
cles and all headers, including titles and bylines, were 
stripped before processing. 
5<x+> = one or more words 
375 
1. For each NP of more than two words, build a candidate pattern of the form "the <x+> 
headnoun." Example: if the NP was "the new Salvadoran government," the candidate pattern 
would be "the <x+> government." 
2. Apply that pattern to the corpus, count how many times it matches an NP. 
3. If possible, grow the candidate pattern by inserting the word to the left of the headnoun, e.g. 
the candidate pattern now becomes "the <x+> Salvadoran government." 
4. Reapply the pattern to the corpus, count how many times it matches an NP. If the new count 
is less that the last iteration's count, stop and return the prior pattern. If the new count is 
equal to the last iteration's count, return to step 3. This iterative process has the effect of 
recognizing compound head nouns. 
Figure 3: EHP Algorithm 
If the NP was identified via the S1 or EHP methods: 
Is its definite probability above an upper threshold? 
Yes: Classify as existential. 
No: Is its definite probability above a lower threshold? 
Yes: Is its sentence-number less than or equal to an early allowance threshold? 
Yes : Classify as existential. 
No : Leave unclassified (allow later methods to apply). 
No : Leave unclassified (allow later methods to apply). 
Figure 4: Vaccine Algorithm 
4.4 Definite-Only List (DO) 
It also became clear that some existentials 
never appear in indefinite constructions. "The 
F.B.I.," "the contrary," "the National Guard" 
are definite NPs which are rarely, if ever, seen 
in indefinite constructions. The chances that 
a reader will encounter "an F.B.I." are slim to 
none. These NPs appeared to be perfect can- 
didates for a corpus-based approach. To locate 
"definite-only" NPs we made two passes over 
the corpus. The first pass produced a list of ev- 
ery definite NP and its frequency. The second 
pass counted indefinite uses of all NPs cataloged 
during the first pass. Knowing how often an NP 
was used in definite and indefinite constructions 
allowed us to sort the NPs, first by the probabil- 
ity of being used as a definite (its definite prob- 
ability), and second by definite-use frequency. 
For example, "the contrary" appeared high on 
this list because its head noun occurred 15 times 
in the training corpus, and every time it was in 
a definite construction. From this, we created a 
definite-only list by selecting those NPs which 
occurred at least 5 times and only in definite 
constructions. 
Examples from the three methods can be 
found in the Appendix. 
4.5 Vaccine 
Our methods for identifying existential NPs are 
all heuristic-based and therefore can be incor- 
rect in certain situations. We identified two 
types of common errors. 
1. An incorrect $1 assumption. When the S1 as- 
sumption falls, i.e. when a definite NP in the 
first sentence of a text is truly referential, the 
referential NP is added to the S1 list. Later, an 
Existential Head Pattern may be built from this 
NP. In this way, a single misclassified NP may 
cause multiple noun phrases to be misclassified 
in new texts, acting as an "infection" (Roaxk 
and Charniak, 1998). 
2. Occasional existentialism. Sometimes an NP 
is existential in one text but referential in an- 
other. For example, "the guerrillas" often refers 
to a set of counter-government forces that the 
reader of an E1 Salvadoran newspaper would 
understand. In some cases, however, a partic- 
ular group of guerrillas was mentioned previ- 
ously in the text ("A group of FMLN rebels 
attacked the capital..."), and later references 
to "the guerrillas" referred to this group. 
To address these problems, we developed a 
vaccine. It was clear that we had a number of in- 
fections in our S1 list, including "the base," "the 
376 
For every definite NP in a text 
1. Apply syntactic RuleOutHeuristics, if any fired, classify the NP as referential. 
2. Look up the NP in the S1 list, if found, classify the NP as existential (unless stopped by 
vaccine). 
3. Look up the NP in the DO list, if found, classify the NP as existential. 
4. Apply all EHPs, if any apply, classify the NP as existential (unless stopped by vaccine). 
5. Apply syntactic RuleInHeuristics, if any fired, classify the NP as existential. 
6. If the NP is not yet classified, classify the NP as referential. 
Figure 5: Existential Identification Algorithm 
individuals," "the attack," and "the banks." 
We noticed, however, that many of these in- 
correct NPs also appeared near the bottom of 
our definite/indefinite list, indicating that they 
were often seen in indefinite constructions. We 
used the definite probability measure as a way 
of detecting errors in the S1 and EHP lists. If 
the definite probability of an NP was above an 
upper threshold, the NP was allowed to be clas- 
sifted as existential. If the definite probability of 
an NP fell below a lower threshold, it was not al- 
lowed to be classified by the S1 or EHP method. 
Those NPs that fell between the two thresholds 
were considered occasionally existential. 
Occasionally existential NPs were handled by 
observing where the NPs first occurred in the 
text. For example, if the first use of "the guer- 
rillas" was in the first few sentences of a text, 
it was usually an existential use. If the first use 
was later, it was usually a referential use be- 
cause a prior definition appeared in earlier sen- 
tences. We applied an early allowance threshold 
of three sentences - occasionally existential NPs 
occuring under this threshold were classified as 
existential, and those that occurred above were 
left unclassified. Figure 4 details the vaccine's 
algorithm. 
5 Algorithm & Training 
We trained and tested our methods on the 
Latin American newswire articles from MUC- 
4 (MUC-4 Proceedings, 1992). The training set 
contained 1,600 texts and the test set contained 
50 texts. All texts were first parsed by SUN- 
DANCE, our heuristic-based partial parser de- 
veloped at the University of Utah. 
We generated the S1 extractions by process- 
ing the first sentence of all training texts. This 
produced 849 definite NPs. Using these NPs as 
Vaccine 
Vaccine~ I 
DO 
EHP I ~' 
/\ 
Unresolved Marked 
referential existential 
definite NPs definite NPs 
Figure 6: Recognizing Existential NPs 
input to the existential head pattern algorithm, 
we generated 297 EHPs. The DO list was built 
by using only those NPs which appeared at least 
5 times in the corpus and 100% of the time as 
definites. We generated the DO list in two iter- 
ations, once for head nouns alone and once for 
full NPs, resulting in a list of 65 head nouns and 
321 full NPs 6. 
Once the methods had been trained, we clas- 
sifted each definite NP in the test set as referen- 
tial or existential using the algorithm in Figure 
5. Figure 6 graphically represents the main el- 
ements of the algorithm. Note that we applied 
vaccines to the S1 and EHP lists, but not to the 
DO list because gaining entry to the DO list 
is much more difficult -- an NP must occur at 
least 5 times in the training corpus, and every 
time it must occur in a definite construction. 
6The full NP list showed best performance using pa- 
rameters of 5 and 75%, not the 5 and 100% used to create 
the head noun only list. 
377 
Method Tested 
0. Baseline 
1. Syntactic Heuristics 
2. Syntactic Heuristics + S1 
3. Syntactic Heuristics + EHP 
4. Syntactic Heuristics + DO 
5. Syntactic Heuristics + S1 + EHP 
6. Syntactic Heuristics + S1 + EHP + DO 
7. Syntactic Heuristics + S1 + EHP + DO + Va(70/25) 
8. Syntactic Heuristics + S1 + EHP + DO + Vb(50/25) 
Recall 
100% 
43.0% 
66.3% 
60.7% 
69.2% 
79.9% 
81.7% 
77.7% 
79.1% 
Precision 
72.2% 
93.1% 
84.3% 
87.3% 
83.9% 
82.2% 
82.2% 
86.6% 
84.5% 
Figure 7: Evaluation Results 
To evaluate the performance of our algorithm, 
we hand-tagged each definite NP in the 50 test 
texts as a syntactically independent existential, 
a semantically independent existential, an asso- 
ciative existential or a referential NP. Figure 8 
shows the distribution of definite NP types in 
the test texts. Of the 1,001 definite NPs tested, 
63% were independent existentials, so removing 
these NPs from the coreference resolution pro- 
cess could have substantial savings. We mea- 
sured the accuracy of our classifications using 
recall and precision metrics. Results are shown 
in Figure 7. 
478 Independent existential, syntactic 48% 
53 Independent existential, semantic 15% 
Associative existential 9% 
::1 Referential 28% 
Total 
Figure 8: NP Distribution 
As a baseline measurement, we considered the 
accuracy of classifying every definite NP as ex- 
istential. Given the distribution of definite NP 
types in our test set, this would result in recall 
of 100% and precision of 72%. Note that we 
are more interested in high measures of preci- 
sion than recall because we view this method 
to be the precursor to a coreference resolution 
algorithm. Incorrectly removing an anaphoric 
NP means that the coreference resolver would 
never have a chance to resolve it, on the other 
hand, non-anaphoric NPs that slip through can 
still be ruled as non-anaphoric by the corefer- 
ence resolver. 
We first evaluated our system using only the 
syntactic heuristics, which produced only 43% 
recall, but 92% precision. Although the syn- 
tactic heuristics are a reliable way to identify 
existential definite NPs, they miss 57% of the 
true existentials. 
6 Evaluation 
We expected the $1, EHP, and DO methods 
to increase coverage. First, we evaluated each 
method independently (on top of the syntac- 
tic heuristics). The results appear in rows 2-4 
of Figure 7. Each method increased recall to 
between 61-69%, but decreased precision to 84- 
87%. All of these methods produced a substan- 
tial gain in recall at some cost in precision. 
Next, we tried combining the methods to 
make sure that they were not identifying ex- 
actly the same set of existential NPs. When 
we combined the S1 and EHP heuristics, recall 
increased to 80% with precision dropping only 
slightly to 82%. When we combined all three 
methods (S1, EHP, and DO), recall increased 
to 82% without any corresponding loss of preci- 
sion. These experiments show that these heuris- 
tics substantially increase recall and are identi- 
fying different sets of existential NPs. 
Finally, we tested our vaccine algorithm to 
see if it could increase precision without sacri- 
ficing much recall. We experimented with two 
variations: Va used an upper definite probabil- 
ity threshold of 70% and ~ used an upper def- 
inite probability threshold of 50%. Both vari- 
ations used a lower definite probability thresh- 
old of 25%. The results are shown in rows 7-8 
of Figure 7. Both vaccine variations increased 
precision by several percentage points with only 
a slight drop in recall. 
In previous work, the system developed by 
Vieria & Poesio achieved 74% recall and 85% 
precision for identifying "larger situation and 
unfamiliar use" NPs. This set of NPs does not 
correspond exactly to our definition of existen- 
tial NPs because we consider associative NPs 
378 
to be existential and they do not. Even so, our 
results are slightly better than their previous re- 
sults. A more equitable comparison is to mea- 
sure our system's performance on only the in- 
dependent existential noun phrases. Using this 
measure, our algorithm achieved 81.8% recall 
with 85.6% precision using Va, and achieved 
82.9% recall with 83.5% precision using Vb. 
7 Conclusions 
We have developed several methods for auto- 
matically identifying existential noun phrases 
using a training corpus. It accomplishes this 
task with recall and precision measurements 
that exceed those of the earlier Vieira & Poesio 
system, while not exploiting full parse trees, ap- 
positive constructions, hand-coded lists, or case 
sensitive text z. In addition, because the sys- 
tem is fully automated and corpus-based, it is 
suitable for applications that require portabil- 
ity across domains. Given the large percentage 
of non-anaphoric discourse entities handled by 
most coreference resolvers, we believe that us- 
ing a system like ours to filter existential NPs 
has the potential to reduce processing time and 
complexity and improve the accuracy of coref- 
erence resolution. 

References 
James Allen. 1995. Natural Language Understand- 
ing. Benjamin/Cummings Press, Redwood City, 
CA. 
Chinatsu Aone and Scott William Bennett. 1996. 
Applying Machine Learning to Anaphora Reso- 
lution. In Connectionist, Statistical, and Sym- 
bolic Approaches to Learning for Natural Lan- 
guage Understanding, pages 302-314. Springer- 
Verlag, Berlin. 
Andrew Kehler. 1997. Probabilistic coreference in 
information extraction. In Proceedings of the Sec- 
ond Conference on Empirical Methods in Natural 
Language Processing (EMNLP-97). 
Christopher Kennedy and Branimir Boguraev. 1996. 
Anaphor for everyone: Pronomial anaphora reso- 
lution without a parser. In Proceedings of the 16th 
International Conference on Computational Lin- 
guistics (COLING-96). 
Shalom Lappin and Herbert J. Leass. 1994. An al- 
gorithm for pronomial anaphora resolution. Com- 
putational Linguistics, 20(4):535-561. 
Joseph F. McCarthy and Wendy G. Lehnert. 1995. 
Using Decision Trees for Coreference Resolution. 
In Proceedings of the l~th International Joint 
Conference on Artificial Intelligence (IJCAI-95), 
pages 1050-1055. 
Ellen F. Prince. 1981. Toward a taxonomy of given- 
new information. In Peter Cole, editor, Radical 
Pragmatics, pages 223-255. Academic Press. 
Brian Roark and Eugene Charniak. 1998. Noun- 
phrase co-occurence statistics for semi-automatic 
semantic lexcon construction. In Proceedings of 
the 36th Annual Meeting of the Association for 
Computational Linguistics. 
R. Vieira and M. Poesio. 1997. Processing defi- 
nite descriptions in corpora. In S. Botley and 
M. McEnery, editors, Corpus-based and Compu- 
tational Approaches to Discourse Anaphora. UCL 
Press. 
