<?xml version="1.0" standalone="yes"?>
<Paper uid="M95-1015">
  <Title>I End of Sentence Detector I TokenizationI IAdwait's TaggerI 'Eric's Tagger1 IX-Tag Tagger ) Tag Voting !Noun Phrase Detector I INamed Entity Tool</Title>
  <Section position="1" start_page="0" end_page="0" type="metho">
    <SectionTitle>
UNIVERSITY OF PENNSYLVANIA :
DESCRIPTION OF THE UNIVERSITY OF PENNSYLVANI A
SYSTEM USED FOR MUC- 6
</SectionTitle>
    <Paragraph position="0"/>
  </Section>
  <Section position="2" start_page="0" end_page="177" type="metho">
    <SectionTitle>
INTRODUCTIO N
</SectionTitle>
    <Paragraph position="0"> Breck Baldwin and Jeff Reynar informally began the University of Pennsylvania's MUC-6 coreference effor t in January of 1995 . For the first few months, tools were built and the system was extended at weekly 'hack sessions.' As more people began attending these meetings and contributing to the project, it grew to include eight graduate students . While the effort was still informal, Mark Wasson, from Lexis-Nexis, became an advisor to th e project. In July, the students proposed to the faculty that we formally participate in the coreference task . By that time, we had developed some of the system's infrastructure and had implemented a simplistic coreference resolution system which resolved proper nouns by means of string matching . After much convincing, the faculty agreed at th e end of July that we could formally participate in MUC-6 . We then began an intensive effort with full-tim e participation from Baldwin and Reynar, and part-time efforts from the other authors . In August we were given permission from Yael Ravin of IBM's Information Retrieval group to use the IBM Name Extraction Module [3] . We were also given access to a large acronym dictionary which Peter Flynn maintains for a world wide web site in Iceland (http ://curia.ucc.ie/info/net/acronyms/acro.html).</Paragraph>
    <Paragraph position="1"> The vast majority of our system was developed in August and September. Our efforts prior to that time were mostly directed towards implementing a parallel-file data structure which allowed new components to be adde d quickly with minimal effort . The ease of incorporating new components was demonstrated by the addition of a ful l syntactic parser two weeks prior to the evaluation . In this data structure, enhancements to the input data, such as tokenization, part-of-speech tags, or parse trees, are stored in separate, aligned files . As a result, building a new module which requires input from earlier components is as simple as loading the files created by those components and performing the necessary processing. The fact that modules further along in the pipeline do not alter the outpu t of earlier components means that output files can be read-only . As a result, the system is afforded a measure o f robustness : if one component fails, further components will not necessarily be crippled and no downstrea m component can alter the output of an earlier component .</Paragraph>
    <Paragraph position="2">  This simple data structure, which was inspired by a pretty-printing convention used by Lexis-Nexis t o display multiple levels of textual annotation, also allowed people to write software in the programming language of their choice. Ultimately, the majority of the code written explicitly for MUC was in Perl 4, but some programs wer e also written in C and several different shell languages. Other system components not developed explicitly for MUC were written in Lisp and C++.</Paragraph>
    <Paragraph position="3"> Despite the advantages of this approach, the parallel-file data structure had some drawbacks . First, because the system was built using many small tools, the number of files grew to be quite large, nearly 100 per article . As a result, disk space became a problem . Second, because of the large number of files and the number of processe s reading each of them, file access time accounted for a significant portion of the time required to run the system . It took approximately 12 minutes to process an average length article when processing was done in batch mode .</Paragraph>
    <Paragraph position="4"> Processing input files in groups allowed the overhead for loading dictionaries and statistical models to be reduce d because it could be averaged over many articles .</Paragraph>
    <Paragraph position="5"> Our coreference resolution system was built from several components, each of which addressed differen t types of coreference . The philosophy behind this methodology was that high precision components could be linked together serially to build an easily extensible, modular system . We focused on building high precision component s on the assumption that many high precision, moderate recall components, when linked together, would yield a system with good overall recall . This goal was met with varying degrees of success . Unfortunately, only one of the three components which posited coreference emerged as being highly precise : the proper name matching component .</Paragraph>
    <Paragraph position="6"> We utilized off-the-shelf components whenever possible . Most of these tools were developed at Penn . As a result, the majority of our efforts went into writing parsers and preprocessing utilities which allowed various pre existing tools to communicate with one another and produce output which could be used by other tools further in th e processing pipeline . Thus, we were freed to spend time developing the task-specific components of the system an d performing data analysis . Although no time was spent developing tools particularly for the MUC task prior t o January, many hours went into developing some of the off-the-shelf components we used, such as Eric Brills part-of-speech tagger [2] and Lance Ramshaw and Mitch Marcus' Noun Phrase Detector [10] . We estimate the tota l number of hours spent on the project itself to be roughly 1800, distributed among the eight graduate students wh o worked on the project. The vast majority of these hours were contributed between the end of July and competitio n week in early October .</Paragraph>
    <Paragraph position="7"> Table 1 shows the performance of our system when simple formatting errors, which hurt performance o n two of the 30 test files, were corrected. Table 2 contains our official system performance figures . Table 3 contain s system performance when optional elements were treated as if required. This set of scores is presented in order to allow comparison between scores for various system components without having to deal with the adjustment to th e number of correct items which results from different components marking coreference between different numbers of optional elements.</Paragraph>
  </Section>
  <Section position="3" start_page="177" end_page="184" type="metho">
    <SectionTitle>
THE SYSTEM
</SectionTitle>
    <Paragraph position="0"> Throughout the system description section, words and phrases which appear in articles will be displayed i n italics . Figure 1 contains a system flowchart . Databases are shown in drums and system modules are shown i n rectangles.</Paragraph>
    <Paragraph position="1"> End of Sentence Detectio n The first step in our processing pipeline is end-of-sentence detection . Sentence boundaries are identified using a maximum entropy model developed explicitly for MUC-6 . This model was built quickly using a general maximum entropy modeling tool which will be discussed in a forthcoming paper [11] . Sentence final punctuation i s defined to include only periods, exclamation points and question marks ; we do not attempt to mark sentence boundaries indicated by semi-colons, commas or conjunctions . Only instances of sentence-final punctuation whic h are immediately followed by white space or symbols which may legitimately follow sentence boundaries, such a s quotation marks, were considered to be potential sentence boundaries . For convenience, we define any sequence of white-space separated tokens to be a word while discussing this stage of processing .</Paragraph>
    <Paragraph position="2"> The maximum entropy model was trained using the dry run and training portions of the MUC-6 coreferenc e annotated data, which included SGML annotated sentence boundaries . The model used binary-valued features of the word to which the putative end-of-sentence marker was conjoined, as well as binary-valued features of the precedin g and following words . These features included whether the word was a corporate designator, such as Corp. or Inc., or an honorific, such as Dr. or Ms. ; whether the word was upper-case ; whether the word was a likely monetary value ; whether the word was likely to be a percentage ; whether the word was a number ; whether the word contained punctuation indicative of ellipsis ; and features indicating whether the word ended in various non-alphanumeric characters.</Paragraph>
    <Paragraph position="3">  We did not subject this component to rigorous testing, but did examine its output for approximately 300 blind test sentences and found that only one error was made . We intend to further refine this component and subject i t to automatic testing against a sentence-detected corpus in the near future .</Paragraph>
    <Paragraph position="4"> Tokenization Once sentence boundaries are identified, tokenization begins . We developed our tokenizer solely for th e MUC coreference task because of specific tokenization requirements . The combination of the character-based nature of the scoring software and the requirements of various tools that punctuation be separated from words forced us t o build a tokenizer which maintains a character offset mapping for all of the tokens in the input messages . A trivia l error in this system caused two of the 30 test messages to be garbled sufficiently that the scorer detected virtually n o correct coreference in them . This is why we are presenting both official and unofficial scores .</Paragraph>
    <Paragraph position="5"> In addition to maintaining the character offset mapping, the tokenizer performs four non-standard tasks . The first is the alteration of headline word capitalization . The Wall Street Journal adheres to standard conventions for capitalization of words in headlines, but since capitalization is an important cue for coreference resolution, w e attempted to eliminate capitalization which resulted solely from these conventions . Headline words which were capitalized in the body of the text anywhere other than sentence-initial position remained capitalized, as did thos e which were frequently capitalized other than in sentence-initial position in the Treebank Wall Street Journal corpu s [8] . All other uppercase words were converted to lowercase .</Paragraph>
    <Paragraph position="6"> The second non-standard task addressed by the tokenizer is the extraction of date information . The dateline field is parsed to determine when each article was written . This information is later used to posit coreference between words or phrases such as today, tomorrow, this week, this year, and dates, such as November 20, 1995.</Paragraph>
    <Paragraph position="7"> The third non-standard component determines whether 's or ' is a genitive marker or part of a company name. When it is actually part of a company name, it does not indicate possession of the following noun phrase . This step was necessary because the part-of-speech taggers and the noun phrase detector required genitive markers t o be tokenized separately, while non-genitive instances of 's or ' were required to remain attached . For instance,  McDonald's, when it refers to the fast-food chain, should be treated as a single token, while Mary's should be separated into two tokens : Mary and 's.</Paragraph>
    <Paragraph position="8"> The final unique task the tokenizer addresses is hyphenated-word splitting . Since coreference is allowed between portions of hyphenated words which are themselves words, such as Apple in the phrase a joint Apple-IBM venture, determining whether a portion of a hyphenated word may participate in coreference is important . The heuristic we use is similar to the one used to determine whether a headline word should be downcased . That is, when one or more of the words which comprise a hyphenated word exists on their own within the article, then th e hyphenated word is split into multiple tokens .</Paragraph>
    <Paragraph position="9"> Unfortunately, because of the nature of the training data used by the noun phrase detector, bare hyphen s cause serious noun phrase detection errors . For simplicity, and because of time limitations, we opted not to retrai n the noun phrase detector. As a result, multiple tokenizations of each article are maintained . In one of the tokenizations, hyphenated words are left unaltered. In the other version, hyphenated words are split into multipl e tokens based on the above criteria. Also, the tokenizer is responsible for maintaining the mapping between these tw o tokenizations so that the output of tools which use different tokenization schemes can be combined .</Paragraph>
    <Paragraph position="10"> Part-of-Speech Tagging Several components of the MUC coreference system, such as the noun phrase detector, require part-of-speech (POS) tags for all of the words in an article . We combined the output of the following three POS tagger s using a simple voting scheme : Eric Brill's Rule Based Tagger version 1 .14 [2], the XTAG tagger, which is an implementation of Ken Church's PARTS tagger [4] and Adwait Ratnaparkhi's Maximum Entropy Tagger [11] . Each of these taggers uses the Penn Treebank tagset [8] .</Paragraph>
    <Paragraph position="11"> These three taggers, which were trained on the Penn Treebank Wall Street Journal corpus, tag pre-tokenized text. The tag actually used by the MUC system is determined by a majority voting scheme, in which a tag is chose n as the &amp;quot;winner&amp;quot; if at least two of the taggers postulate it. In the rare event that all three taggers disagree, the system uses the tag assigned by the maximum entropy tagger . In most cases, the majority voting scheme eliminates errors that are esoteric to a single tagger, and should therefore perform better than any single tagger . We did not have time to empirically verify this hypothesis, but intend to do so in the future . We may also improve upon the voting model by incorporating information regarding which tagger proposed each tag .</Paragraph>
    <Paragraph position="12"> Basal Noun Phrase Detection To identify noun phrases, the system uses Lance Ramshaw and Mitch Marcus' basal noun phrase detecto r [10] . Basal noun phrases are those noun phrases in the lowest level of embedding in the Penn Treebank' s annotations. Intuitively, they are the smallest noun phrases in a parse . For example, chief executive officer and International Business Machines are both basal noun phrases, but chief executive officer of International Business Machines is not, since it contains nested noun phrases . Ramshaw and Marcus' noun phrase detector is based on Eri c Brill's work on learning transformational rules for part-of-speech tagging . It was trained using a section of the tagged and parsed Treebank Wall Street Journal corpus disjoint from the MUC-6 test data .</Paragraph>
    <Paragraph position="13"> We postprocess the output of their tool to make it more appropriate for the coreference task . For instance, i t brackets noun phrases containing genitives in the following way : [Noun Phrase 1] ['s Noun Phrase 2] . But, we prefer [Noun Phrase 1] 's [Noun Phrase 2] since it is more appropriate for further processing steps . In addition, we manually added some transformations to the set learned from the treebank . These transformations generalized on learned ones . For instance, rules were learned which involved days of the week, but due to sparsity of training data, they were learned only for a subset of the seven days of the week . We manually added the missing cases . We did no t independently measure the performance of their tool using this modified rule set, but may do so in the future .  We experimented with various knowledge sources during system development, including WordNet [9], th e XTAG morphological analyzer [6], Roget's publicly available 1911 thesaurus, the Collins dictionary, a version of the American Heritage dictionary for which the University of Pennsylvania has a site license and the Gazetteer . Only WordNet, the XTAG morphological analyzer and the Gazetteer were used in the final system .</Paragraph>
    <Paragraph position="14"> We extracted a geographic name database from a publicly available version of the Gazetteer which we downloaded from the Center for Lexical Research. This database contains names of continents, islands, island groups , countries, provinces, cities and airports . This information is used when performing type checking prior to positin g coreference between entities .</Paragraph>
    <Paragraph position="15"> The XTAG morphology database [6] was originally extracted from the 1979 edition of the Collins English Dictionary and the Oxford Advanced Learner's Dictionary of Current English, and then edited and augmented by hand . It contains approximately 317,000 inflected items, along with their root forms and inflectional information, such as case, number and tense. Thirteen parts of speech are differentiated : noun, proper noun, pronoun, verb, verb particle, adverb, adjective, preposition, complementizer, determiner, conjunction, interjection, and noun/verb contraction . Nouns and verbs are the largest categories, with approximately 213,000 and 46,500 inflected forms, respectively . Tagging for Gender, Number and Animacy To resolve pronouns which typically select for a gendered antecedent as well as those that typically selec t for an animate antecedent, gendered or non-gendered, the WordNet 1 .5 lexical database [9] for nouns is used to tag each potential antecedent with respect to these semantic features . In addition, rudimentary morphological analysis o f the head of a noun phrase is performed and several databases are consulted to determine whether a particular nou n phrase refers to a male, a female, or a person of either gender . Also, some singular count nouns, such as committee, may be the antecedents of plural pronouns . WordNet is also consulted to tag such nouns as possibly having sets of individuals as their referent.</Paragraph>
    <Paragraph position="16"> WordNet's noun database is organized as an inheritance lattice . For example, the entry for man is linked to daughter nodes which include the entries bachelor, boyfriend, eunuch, etc. Assuming that a semantic feature such as maleness generally will propagate from a parent in the hierarchy to its children, one can test the gender of a give n noun by examining its ancestors . If one of the ancestors is the entry male, for example, it may be concluded that th e word itself typically denotes an entity which is male . Similarly, the WordNet entry socialgroup tends to subsume nouns which can have groups of individuals as their referents.</Paragraph>
    <Paragraph position="17"> Unfortunately, the WordNet taxonomy is more like a tree than a lattice, so that many useful multipl e inheritance links do not exist. For example, the entry for uncle is not a descendant of the entry for man, although an uncle is clearly a type of man . Additionally, as with any semantic inheritance hierarchy, not all features are alway s passed down from parent to child, so that strictly monotonic reasoning is not valid .</Paragraph>
    <Paragraph position="18"> To ameliorate these deficiencies and complications, the query to WordNet takes the form of a Boolean quer y about the ancestors of a given word entry . For example, an OR operator is used to tag as male words which ar e descendants of either the male node or the kinsman node, which subsumes uncle. This supplants the missin g inheritance link, which would be needed in a complete semantic taxonomy, between male and kinsman. To prune ou t descendants of an entry such as man which do not inherit the semantic feature of maleness, an AND NOT operator can be used to exclude subclasses of the class of descendants of male . Additionally, to circumvent problems with solely relying on the Boolean query, a word's definition is also examined in a rudimentary way, to check for ke y words that indicate semantic features of the potential referents of this word, such as the word someone, which suggests a human referent.</Paragraph>
    <Paragraph position="19"> For polysemous words, WordNet may give conflicting evidence because of the word's multiple senses . For example, end is judged as potentially compatible with a human referent, because an end is a type of football player . But in most contexts, this sense of end will be wrong and this word should not be considered as the potentia l antecedent for a pronoun such as he.</Paragraph>
    <Paragraph position="20">  Therefore, the evidence from WordNet is weighted on a scale of plausibility . The evidence for uncle is considered more plausible than that for end because both senses of uncle in WordNet have the entry person among their ancestors. On the other hand, only one of the thirteen senses for end has person as its ancestor. Moreover, no t all of the senses of end are equally likely to occur . The WordNet semantic concordance provides frequency information from a fraction of the Brown Corpus for senses of end and other words in the noun database. These counts can be used to estimate the probabilities of WordNet word senses . When no data is available from the semantic concordance for some senses of a word, the gaps in frequency are smoothed . If no data is available for any sense of the word, the uniform distribution is assumed .</Paragraph>
    <Paragraph position="21"> The evidence from WordNet is then weighted according to how likely it is that the sense for which th e evidence is obtained is the correct sense of the word seen in the input file . A more sophisticated approach would involve using word-sense disambiguation techniques to guess the correct sense of the word, and then only quer y WordNet about that particular sense. However, the method employed in the current system is able to discriminat e reliably on a coarse level between cases like end and uncle. A weight of 1 .0 is assigned to the person feature for uncle, whereas only 0.024 is assigned to this feature in end.</Paragraph>
    <Paragraph position="22"> As a second source of evidence about the gender or animacy of noun phrase referents, two tables of gendered first names, compiled by Mark Kantrowitz and Bill Ross and freely available from the Computing Research Laboratory of New Mexico State University, are consulted . The table of first names overlaps with place names and time words . For example, Canada and Tuesday are women's names . In such cases, the evidence from the table i s discarded . This evidence is weighted separately from the WordNet look-up results .</Paragraph>
    <Paragraph position="23"> Finally, a rough analysis of the suffix morphology of the word is undertaken . Nouns ending in &amp;quot;-man &amp;quot; which do not end in &amp;quot;-woman&amp;quot; tend to denote male humans . However, due to the inherent gender bias of language , words such as chairman can also be used to refer to women . Hence such words also count as evidence of a female referent, but to a lesser degree. This results in both the male weight and the female weight being set to non-zero values. The difference in weighting between the two is currently based on intuition, though corpus methods migh t yield a more exact estimate of how much weight to give the female reading based on how often such words are actually used to refer to women .</Paragraph>
    <Paragraph position="24">  It is often used anaphorically in Wall Street Journal Text . Nonetheless, identifying instances of pleonastic it, which do not corefer, is still significant. The system identifies these instances of it by scanning tagged text and applying partly syntactic and partly lexical tests . Most of these tests are described in [7], but some additional test s were added to increase coverage . The fifteen rules used to detect pleonastic it are shown below in table 4 . Part of speech tags follow words and a slash, and are specified using the Penn Treebank tagset . Disjunctions are indicated using a vertical bar, (I), and optional elements are surrounded by brackets, ([]) . S abbreviates sentence ; NP mean s noun phrase; and VP stands for verb phrase . We abbreviate CA for comparative adjectives, such as larger or smaller; SA for superlatives, such as greatest or largest; MA for modal adjectives, such as necessary or uncertain; MV for modal verbs, like could or will; CV for cognitive verbs, such as recommended or hoped; and CADV and SADV for comparative and superlative adverbs.</Paragraph>
    <Paragraph position="25">  It's performance is shown in table 5 . Our first attempt at a coreference system, La Hack 1, posited coreference between identical upper case words in the text, and was written to test the validity of the system's SGML annotatio n and to test the tokenizer. La Hack 2 was written to do more sophisticated string matching . It uses several knowledge sources, including the IBM Name Extraction Module, and a simple unification system to produce coreference chains . The knowledge sources are used to determine whether an entity is of type person, place, corporation or other . Most of the entities which La Hack 2 annotates are proper nouns, but the date information extracted by the tokenizer i s used here as well. The majority of the strings annotated are noun phrases detected by the noun phrase detector, bu t some sub-noun phrase units are annotated as well. Proper nouns which are portions of longer noun phrases may b e annotated. For example, Apple in the phrase Apple stock prices would be annotated if there were other references to Apple in the article.</Paragraph>
    <Paragraph position="26"> La Hack 2 makes four passes through each article . On the first, it builds coreference chains containin g alternate forms of corporate and person names as identified by the Name Extraction Module. These variant references include references to people by first name only, last name only, last name and an honorific, and references whic h omit middle names. For instance, General Colin Powell could be referred to as General Powell, Colin, Powell, Mr. Powell and so forth . Variant corporate names may be references which exclude corporate designators, use acronym s or omit a company's industry. For example, Apple Computer Inc. might be referred to as Apple, Apple Inc., etc. The next processing step looks for date matches, and those alternate forms not identified by the IBM tool .</Paragraph>
    <Paragraph position="27"> The third step looks for upper case string matches which are not variant name references or which do not contai n corporate designators or honorifics. Product names, some acronyms and miscellaneous other upper case words are entered into coreference chains in this stage . The final stage is an upper case substring match which is targeted at finding coreference chains which were missed by the named entity tool and the other stages as well .</Paragraph>
    <Paragraph position="28"> The purpose of the simple type system is mainly to prevent coreference chains from being created by th e substring matching stage which contain substrings of different types . For instance, Apple is a substring of Apple CEO John Sculley, but they cannot be coreferent since John Sculley is a person and Apple is a corporation .</Paragraph>
    <Section position="1" start_page="184" end_page="184" type="sub_section">
      <SectionTitle>
Parser
</SectionTitle>
      <Paragraph position="0"> The parser we use has been developed over the past 6 months by Michael Collins, and is a continuation of the work on prepositional phrase attachment described in [5] . It was trained on 33000 sentences from the Wall Stree t Journal Treebank [8] . As yet no extensive performance tests have been made, but both recall and precision on labele d edges is over 80%. The parser was used to spot syntactic patterns which signaled coreference of noun phrases withi n sentences, such as appositive relations and predicate nominative constructions . The performance of this component is shown in table 6 .</Paragraph>
      <Paragraph position="1"> Given a maximal noun phrase, we find the head non-recursive noun phrase through a left-recursive descent of the parse tree. For example Fred Bloggs, president of ACME, who was elected yesterday would be reduced to Fred Bloggs. In addition, if either of the noun phrases involves conjunction, as in president of General Motors and forme r CEO of Ford, both minimal noun phrases, president andformer CEO would be recovered .</Paragraph>
      <Paragraph position="2"> We mark one noun phrase, called NP1, as being coreferent with a second noun phrase, NP2, because of a n appositive relationship if NP1 is the head of a parent noun phrase, and NP2 is also a direct descendant of this paren t noun phrase. For example, in the phrase John Smith, president of ACME, a former worker at Eastern, John Smit h is coreferent with both president and a former worker. Note that the parser incorporates punctuation into the statistical model, so a comma between two noun phrases is seen as a strong indication of an appositive relationship . The Wall Street Journal uses constructions similar to appositives to indicate relationships other tha n coreference. For example, such constructions are used with place names, such as Frankfurt, Germany or Smith Barney, Harris Upham &amp; Co . , New York ; ages, such as Al Bert, 49; and dates, such as March 31, 1989. These constructions are a source of error in appositive recognition . In addition, the parser confuses some instances o f conjunction with appositives. For this reason, semantic filtering is required to raise precision . We found that the following strategy worked remarkably well : given the two proposed minimal noun phrases, if the first one has a capitalized head, and the second head begins with a lower-case letter, accept the pair as coreferent . Note that this would deal correctly with all the above examples . A few additional cases were caught by allowing pairs where the first head word was on a list of honorifics, such as president, chairman, journalist, or CEO, and the second head was capitalized. This heuristic correctly handles cases such as ACME 's president, Bill Jones . Also, a later processin g stage removes indefinite cases from those proposed as appositives . While not appearing in the final output, thes e cases are used to aid in positing other types of coreference .</Paragraph>
      <Paragraph position="3"> Definite cases of predicate nominative constructions are also markable . As a result, syntactic patterns of th e type 'NP is NP' are also recognized, as are constructions involving the verbs remain or become, which function in a similar way to be. These could appear in sentential clauses or in relative clauses, such as Fred Flintstone, who is Wilma's husband. As is the case with appositives, indefinites are filtered from the final output, but are marked and used in later processing .</Paragraph>
      <Paragraph position="4"> Several verbs function similarly to become and remain, but subcategorize for a prepositional phrase headed by as, with the object of this prepositional phrase being coreferent with the subject of the verb. A list of these verbs , including serve, work, continue and resign, was compiled and these patterns were used as well .</Paragraph>
      <Paragraph position="5"> It was found that most verb phrases, regardless of the verb head, which take both a noun phrase, NP1, and a prepositional phrase headed by as with an object, NP2, imply coreference between NPI and NP2 . This was extended to include patterns of the form 'verb npl (to be np2) .' Some examples are shown below. Underlined entities are coreferent.</Paragraph>
      <Paragraph position="6"> 18 5 Mr. Casey succeeds M. James Barrett, 50, as president of Genetic Therapy But the mainstream civil-rights leadership generally avoided the rhetoric of &amp;quot;law and order,&amp;quot; regarding it. as a cote for keeping blacks back We consider our Butthead to be an endearing. fun-loving guy,&amp;quot; a spokesman says In addition, patterns were implemented to identify phrases containing monetary figures in which alternate representations of the amount are present. Some such phrases are : $53 , or 20 cents a share, 23 billion marks (1 5 billion dollars) and profits climbed to 11 million dollars .</Paragraph>
      <Paragraph position="7">  Parsing enables regular expressions to be written which apply to trees rather than surface text. These patterns are simpler and more intuitive than equivalent surface regular expressions . It is trivial to add new patterns to the system, since the parser has effectively abstracted away many of the complications of the surface text . While regular expressions could catch many of the phenomena we have described, they will become increasingly comple x as they attempt to capture long range dependencies in the text and will also become increasingly inaccurate. Bride of CogNIA C Resolution of pronouns and lower-case anaphors was handled by a program called Bride of CogNIAC, whic h is an extension of CogNIAC, [1] . CogNIAC was designed to perform pronominal resolution in highly ambiguou s contexts and is distinguished from other approaches to pronominal resolution in the following ways. First, it was designed to have high precision, rather than high recall . Second, it ranks the relative salience of an anaphor' s candidate antecedents in a partial order rather than a total order . This means that two candidate antecedents can be equally salient. And, third, it requires that there be a unique antecedent for an anaphor . Uniqueness is achieved b y eliminating competing antecedents using semantic information or by preferring some candidate antecedents ove r others. CogNIAC will not commit to a resolution if a unique referent cannot be found .</Paragraph>
      <Paragraph position="8"> Bride of CogNIAC also handles lower-case definite descriptions using various knowledge sources to do semantic classification of noun phrases into categories such as person, male, female, place, thing, singular an d plural. It also employs the pleonastic-it filter described above and a quoted speech component not present i n CogNIAC . Bride of CogNIAC performs resolution on basal noun phrase detected and part-of-speech tagged text . It also relies on proper noun anaphora information provided by La Hack 2 and syntactic anaphora information posite d by the parser . System performance prior to running Bride of CogNIAC, the last component which posits coreference ,  in table 8 . It equates markables which share a common head noun using various metrics of similarity . The biggest 18 6 difficulty is to prevent Bride of CogNIAC from marking too many things as coreferent. As a result, various heuristics are used to reduce the number of entities marked . For example, coreference is not posited if:  The second and final task addressed by Bride of CogNIAC is the resolution of pronominals and words whic h behave like pronominals, such as company . Performance for this component alone is shown in table 9 . Overall official results are shown in table 2. Overall unofficial results are shown in table 1 .</Paragraph>
      <Paragraph position="9">  We were disappointed by the performance of the pronoun resolution component . In examining the outpu t briefly, the mistakes made were due to knowledge-base failures and bugs more than issues inherent to the pronou n resolution algorithm . This is clearly an aspect of the task where better knowledge representation would improv e system performance .</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>