<?xml version="1.0" standalone="yes"?>
<Paper uid="H93-1025">
  <Title>HEURISTICS FOR BROAD-COVERAGE NATURAL LANGUAGE PARSING</Title>
  <Section position="1" start_page="0" end_page="0" type="metho">
    <SectionTitle>
HEURISTICS FOR BROAD-COVERAGE
NATURAL LANGUAGE PARSING
</SectionTitle>
    <Paragraph position="0"/>
  </Section>
  <Section position="2" start_page="0" end_page="0" type="metho">
    <SectionTitle>
ABSTRACT
</SectionTitle>
    <Paragraph position="0"> The Slot Grammar system is interesting for natural language applications because it can deliver parses with deep grammatical information on a reasonably broad scale. The paper describes a numerical scoring system used in Slot Grammar for ambiguity resolution, which not only ranks parses but also contributes to parsing efficiency through a parse space pruning algorithm. Details of the method of computing parse scores are given, and test results for the English Slot Grammar are presented.</Paragraph>
  </Section>
  <Section position="3" start_page="0" end_page="127" type="metho">
    <SectionTitle>
1. INTRODUCTION
</SectionTitle>
    <Paragraph position="0"> As everyone who has tried it knows, the hardest part of building a broad-coverage parser is not simply covering all the constructions of the language, but dealing with ambiguity.</Paragraph>
    <Paragraph position="1"> One approach to ambiguity resolution is to &amp;quot;understand&amp;quot; the text well enough - to have a good semantic interpretation system, to use real-world modeling, inference, etc. This can work well in small domains, and it is, in this author's opinion, ultimately necessary for the highest quality of natural language processing in any domain; but it is probably not feasible on a broad scale today. So some kind of heuristic method is needed for disambiguation, some way of ranking analyses and choosing the best. Even in the ideal model of human language processing (which would use a great deal of knowledge representation and inference), ranking heuristics seem appropriate as a mechanism since humans must work with incomplete knowledge most of the time.</Paragraph>
    <Paragraph position="2"> Two major questions that can be asked about a heuristic method for ambiguity resolution are these: 1. What level of representation is used for disambiguation and is involved in the statements of the heuristic rules lexical/morphological, surface syntactic, deep syntactic, or logical/semantic? 2. Where do the heuristic rules come from? Are they largely created through human linguistic insight, or are they induced by processing corpora? This paper describes the heuristic method used in the Slot Grammar (SG)system \[10, 11, 13, 16, 17\] for ambiguity resolution - the SG parse scoring system. This scoring system operates during parsing (with a bottom-up chart parser), assigning real number scores to partial analyses as well as to analyses of the complete sentence. The scores are used not only for ranking the final analyses but also for pruning the parse space during parsing, thus increasing time and space efficiency.</Paragraph>
    <Paragraph position="3"> The level of representation being disambiguated is thus the level of SG parses. SG parse structures are dependencyor head-oriented, and include, in a single tree, both surface structure and deep syntactic information such as predicate-argument structure, remote dependencies, control information, and unwinding of passives. 1 SG parse structures also include a choice of word senses. The extent to which these represent semantic sense distinctions depends on the lexicon. The SG system is set up to deal with semantic word-sense distinctions and to resolve them by doing semantic type-checking during parsing. However, in the lexicon for ESG (English Slot Grammar), nearly all word sense distinctions are a matter of part of speech or syntactic slot frame. Some semantic types are shown in the lexicon and are used in parsing, but generally very few. Thus one would say that ESG parse structures are basically syntactic structures, although the deep information like argument structure, passive unwinding, etc., counts for &amp;quot;semantics&amp;quot; in some people's books.</Paragraph>
    <Paragraph position="4"> Where do the SG scoring rules come from - human linguistic insight or induction from corpus processing? The score of an SG parse, which will be described in Section 4, is the sum of several components. Most of these come completely from human linguistic insight, though some of them get their numeric values from corpus processing. In the tests reported in the final section, only the &amp;quot;linguistic-insight&amp;quot;rules are used. Some previous tests using the corpus-based heuristic rules together with the main SG heuristic rules showed that the former could improve the parse rate by a few percentage points. It is definitely worth pursuing both approaches, and more work will be done with a combination of the two.</Paragraph>
    <Paragraph position="5"> I No attempt is made to resolve quantifier scoping in SG parses, although there is a post-processing system that produces a logical form with scope resolution for quantifiers and other &amp;quot;focalizers&amp;quot;\[12\]. Anaphora resolution \[8, 9\] is also done by post-processing SG parses.</Paragraph>
    <Paragraph position="6">  In the next section we give a brief overview of Slot Grammar.</Paragraph>
    <Paragraph position="7"> In Section 3 we describe the scoring system generally and its use in parse space pruning, and in Section 4 we give some details of the computation of scores. Finally, Section 5 presents the results of some tests of ESG.</Paragraph>
  </Section>
  <Section position="4" start_page="127" end_page="127" type="metho">
    <SectionTitle>
2. OVERVIEW OF SLOT GRAMMAR
</SectionTitle>
    <Paragraph position="0"> The slots that figure in Slot Grammar rules and parsing come in two varieties: complement slots and adjunct slots. Analysis is word-oriented, and slots are associated with word senses. The complement slots for a word sense are associated with it in the lexicon. The adjunct slots depend only on the part of speech of the word sense and are listed for that part of speech in the grammar. Slots have names like subj and obj and should be thought of basically as syntactic relations, though complement slot frames in the lexicon can be viewed as corresponding to arguments in logical form.</Paragraph>
    <Paragraph position="1"> The notion that a phrase fills a slot of a word (sense) is primitive in the grammar, and the conditions under which this can happen are given by the slot-fillerrules. Grammatical analysis of a phrase consists basically of choosing, for each word of the phrase, (1) a word sense, (2) a feature structure, and (3) filler subphrases for its slots. A slot is obligatory or optional according as it must be, or need not be, filled in order for the analysis to be complete. Adjunct slots are normally optional.</Paragraph>
    <Paragraph position="2"> A complement slot can be filled at most once, but adjunct slots can, by default, be filled multiply.</Paragraph>
    <Paragraph position="3"> The parser works bottom-up, beginning with one-word phrases and attaching other phrases as left and right modifiers as they can fill slots. As a phrase is built up in this way, it retains a distinguished head word, and the slots associated with this head word are considered slots of the phrase.</Paragraph>
    <Paragraph position="4"> An example of a slot-filler rule is the following for the subject slot (in simplified form): subj ~ f ( noun( nom, N um) ) &amp; h f ( verb( fin( N urn) ) ).</Paragraph>
    <Paragraph position="5"> A goal f(Feat) on the right hand side of a filler rule tests that the feature structure of the filler phrase matches Feat. A goal hf(Feat) tests the feature structure of the higher phrase- the phrase (with possibly other modifiers attached) with which the slot is associated. The SG formalism includes a rich set of special predicates like f and hf that can be used for examining any aspects of the filler phrase and higher phrase for a slot filling.</Paragraph>
    <Paragraph position="6"> Slot-filler rules normally do not constrain left-to-right ordering of the phrases involved. Instead, there are modularly stated ordering rules, which are applied as constraints in parsing after slot-filler rules apply.</Paragraph>
    <Paragraph position="7"> Generally, there is a modular treatment of different grammatical phenomena in a Slot Grammar. There are separate rule systems not only for slot-filling and ordering, but also for coordination, unbounded dependencies, obligatory slots, adjunct slot declaration, &amp;quot;or-slots&amp;quot;, punctuation, and parse scoring. All these rule types make use of the same system of special predicates (mentioned above for slot-filler rules) for examining the phrases involved in slot filling. Modularization of the rule system makes large grammars more manageable and also makes it easier to adapt a grammar for one language to another language.</Paragraph>
    <Paragraph position="8"> There are currently Slot Grammars (in various states of completeness) for English, German, Spanish, Danish, Norwegian, and Hebrew. A great deal of attention has been paid to the development of a large, language-universal component of the system, the Slot Grammar shell. For a particular language, the shell represents roughly 65% of the rules/facts, not counting lexicons. All of the rule types mentioned above have part of their treatment in the shell, but there are especially large language-universal components for coordination, unbounded dependencies, punctuation, and parse scoring. Nevertheless, for all of these, there can be rules in the language-specific grammar that override or augment the language-universal rules.</Paragraph>
    <Paragraph position="9"> The lexicon for ESG consists of a hand-coded portion for approximately 6,000 lemmas (basically most frequent words), plus a large back-up lexicon of approximately 60,000 lemmas derived from UDICT \[1, 7\] and other sources. Mary Neff is working on improvements of the large ESG lexicon through extraction from standard dictionaries.</Paragraph>
    <Paragraph position="10"> Slot Grammars are used for source analysis in the MT system LMT \[14, 15\].</Paragraph>
    <Paragraph position="11"> For a more detailed description of current version of the SG system, see \[16, 17, 18\]. In this paper we concentrate on the scoring system, in its latest form.</Paragraph>
  </Section>
  <Section position="5" start_page="127" end_page="128" type="metho">
    <SectionTitle>
3. SCORING AND PARSE SPACE
PRUNING
</SectionTitle>
    <Paragraph position="0"> During parsing, each analysis P of a subphrase is assigned a real number score(P). A larger number represents a worse score. As described in the next section, most of the ingredients that go into scores are positive numbers that are like penalties for unusual structure, and total scores are normally positive.</Paragraph>
    <Paragraph position="1"> Parse space pruning involves comparison of scores of partial analyses and pruning away analyses that have relatively bad scores; but the comparisons are made only within certain equivalence classes of analyses. Two partial analyses are equivalent when they have the same boundaries, the same head word, and the same basic feature. For most categories, the basic feature is just the part of speech, but for verbs a finer distinction is made according to the inflection type (finite,  infinitive, etc.) of the verb. The notion of equivalence is loosened in certain ways for coordinated phrases that will not be described here.</Paragraph>
    <Paragraph position="2"> Pruning is done as follows. Suppose P is a new, candidate partial analysis obtained in parsing. Let compar(P) denote the set of existing partial analyses that are equivalent to P (not including P itself). Because of previous pruning, all members of compar(P) have the same score; call this number scompar(P). (If corn/mr(P) = 0 then consider scompar(P) = +oo.) The system stores this best score scompar(P) for the equivalence class of P in a way that can immediately be computed from P without searching the chart.</Paragraph>
    <Paragraph position="3"> Now three things can happen: (1) If score(P) &gt; scornpar(P), then P is discarded. (2) If score(P) = scompar(P), then P is simply added to the chart. (3) If score(P) &lt; scorapar(P), then P is added to the chart and all members of compar(P) are discarded.</Paragraph>
    <Paragraph position="4"> This parse space pruning can be turned on or off at run time, but scoring is done in any case, and final parses are ranked by the scoring system whether pruning is on or off. Generally, parse space pruning is crucial to the running of the system for large sentences because of space and time problems if it is not used. When pruning is turned on, there are generally very few final parses obtained for a sentence - on average about 1.3 (per successfully parsed sentence).</Paragraph>
    <Paragraph position="5"> When parsing of a sentence fails, the system pieces together a &amp;quot;fitted parse&amp;quot; in a manner somewhat similar to that in \[5\]. The scores obtained for partial analyses figure heavily in choosing the pieces for this result.</Paragraph>
  </Section>
  <Section position="6" start_page="128" end_page="130" type="metho">
    <SectionTitle>
4. COMPUTATION OF SCORES
</SectionTitle>
    <Paragraph position="0"> The score of a partial analysis is obtained incrementally in building up the analysis. The initial score, for a one-word analysis, is associated with the word sense, and the score is incremented whenever a slot is filled or a coordinate structure is formed. All in all, there are eight main components of the total score (the score is the sum of them). We first list them with a brief description and then discuss them individually.</Paragraph>
    <Paragraph position="1"> The list is in decreasing order of&amp;quot;importance&amp;quot; - the amount of effort put into the rules/data for the component and roughly the current contribution of the component to successful parsing.</Paragraph>
    <Paragraph position="2"> Components 1, 2, 3, 4, 5, 8 are totally &amp;quot;human-coded&amp;quot;, and components 6, 7 get their data from corpus processing.</Paragraph>
    <Paragraph position="3"> Most of the heuristics described in \[4\] are covered by these rules (and were developed independently).</Paragraph>
    <Paragraph position="4">  1. SlotPref. Computed when a modifier is attached by filling a slot. Measures the preference for using that slot vs. other slots for attaching the given modifier to the given higher phrase.</Paragraph>
    <Paragraph position="5"> .</Paragraph>
    <Paragraph position="6"> 3.</Paragraph>
    <Paragraph position="7"> 4.</Paragraph>
    <Paragraph position="9"> ParallelCoord. Favors parallelism in coordination.</Paragraph>
    <Paragraph position="10"> CloseAttach. Favors close attachment.</Paragraph>
    <Paragraph position="11"> PhrasePref. Tests the characteristics of a &amp;quot;completed&amp;quot; phrase - a phrase that becomes a modifier or is taken as an analysis of the complete input segment. Similar to SlotPref, but independent of the slot.</Paragraph>
    <Paragraph position="12"> WordSensePref. Favors one sense of a word over other senses.</Paragraph>
    <Paragraph position="13"> HeadSlotFiller. Used, like SlotPref, when a slot is filled, but tests for specific choices for the head words of the higher phrase and the filler phrase, as well their parts of speech, and tests only these things.</Paragraph>
    <Paragraph position="14"> POSPref. Similar to WordSensePref, but tests only for part of speech.</Paragraph>
    <Paragraph position="15"> LexSlotPref. Another score associated with filling a given slot, but coded in the lexicon in a given slot frame and can test semantic type conditions on the filler. In the following more detailed description of the scoring components, we will use &amp;quot;XSG&amp;quot; to refer to the language-specific part of the Slot Grammar of language X. Thus the rules for grammar of X reside in both the SG shell and XSG.</Paragraph>
    <Paragraph position="16"> SlotPref The rules for SlotPref are coded in both the shell and XSG. The default score, given in the shell, is +I for an adjunct slot and 0 for a complement slot, so that complement slots are preferred over adjuncts.</Paragraph>
    <Paragraph position="17"> For an example, consider the sentence John sent the file to Bill. The PP to Bill can attach tofile by an adjunct slot or to sent by a complement slot (its indirect object), but the default SlotPrefscore is 1 for the adjunct and 0 for the complement, 2 so the analysis with the complement wins and the other is pruned away. 3 Slot-scoringrules in XSG can override the default. Currently, out of a total of 678 rules of various types in ESG, 216 are slot-scoring rules. Most of the day-to-day effort in improving ESG consists of work on these scoring rules.</Paragraph>
    <Paragraph position="18"> A slot-scoring rule is of the form: Slot + E ( 4-- Body).</Paragraph>
    <Paragraph position="19"> E is normally a real number and is the contribution of this rule to SlotPref. The Body, if present, can contain special predicates like those mentioned in Section 2 for slot-filler 2Actually, a slot-scoring rule in ESG gives the adjunct a score of 2 in this instance.</Paragraph>
    <Paragraph position="20">  rules that test any characteristics of the filler phrase and the higher' phrase.</Paragraph>
    <Paragraph position="21"> Two examples of slot-scoring rules in ESG are as follows (given: in simplified form).</Paragraph>
    <Paragraph position="22"> ndet + 0 ~-- hf(noun(T, ., *))&amp;(T = chiT = gerund).</Paragraph>
    <Paragraph position="23"> vadv + 0 /(noun(*, *, *))&amp;,t(tm).</Paragraph>
    <Paragraph position="24"> The first says that the determiner slot for nouns is rewarded if the higher noun is a common noun or a gerund. The second says that the vadv slot for verbs is rewarded when it is tilled by a time noun phrase. The special goal st(Type) says that Type is a semantic type of the (sense of) the filler phrase, and ira is a semantic type used in ESG for time words.</Paragraph>
    <Paragraph position="25"> A slot-scoring rule might be used to penalize the use of a complement slot under certain conditions, by assigning it a score higher than the default 0. An example of this in ESG is</Paragraph>
    <Paragraph position="27"> Here comp(binf) is a verb complement slot filled by a bare infinitive VP. This is penalized if the head word of the filler is also a noun and the higher verb has (so far) no right modifiers.</Paragraph>
    <Paragraph position="28"> This is to discourage use of the cornp(binf) analysis when the phrase may really be an NP, maybe an object. Several of the slot-scoring rules in ESG involve checks on existing alternative analyses of words, as in this example. It is quite useful to have such heuristics because of the great ambiguity of English in part of speech of words.</Paragraph>
    <Paragraph position="29"> Slot-scoring may use conditions on punctuation. For example, for the slot nprep that finds adjunct PP modifiers of nouns, we might have: nprep + 2 ~ ~sep(nil)&amp;~hrmodf( *, prep(*, *, .)).</Paragraph>
    <Paragraph position="30"> This penalizes nprep if the separator is not nil (say, if there is a comma separator) and there is no other PP postmodifier already. Thus, in an example like John noticed his neighbor, from across the street, there will be a preference for the PP to modify noticed instead of neighbor.</Paragraph>
    <Paragraph position="31"> ParalleICoord Most of the rules for this score component are in the shell. Parallelism in coordinate structures is measured by similarity of the conjuncts with respect to several different characteristics. Explicitly, when a coordinate structure is formed, the increment in the total score due to the ParallelCoord component is currently given by a formula (slightly simplified): P F ea + P Frarne-F P Sense 4- P M ods + P Len-F PConj + PXSG.</Paragraph>
    <Paragraph position="32"> Here the first five ingredients measure similarity of the conjuncts with respect to the following: (1) PFea: feature structures; (2) PFrame: complement slot frames; (3) PSense: word senses; (4) PMods: modifier list lengths; (5) PLen: word list lengths.</Paragraph>
    <Paragraph position="33"> The ingredient PConj tests for certain characteristics of the conjunction itself (which can include punctuation).</Paragraph>
    <Paragraph position="34"> The ingredient PXSG represents a contribution from language-specific coordination rules in XSG.</Paragraph>
    <Paragraph position="35"> CloseAttach This score is essentially the same as that developed by Heidorn \[3\], and is designed to favor close attachment, although SlotPrefand ParallelCoordcan easily override it.</Paragraph>
    <Paragraph position="36"> For a phrase P, the default for CloseAttaeh(P) is defined recursively as the sum of all terms</Paragraph>
    <Paragraph position="38"> where MOd varies over the modifiers of P. (One need not state the base of this recursive formula separately, since one arrives eventually at phrases with no modifiers, and then the sum over the empty list is understood to be zero.) The factor 0.1 used in the recursive formula is the default, and it can be overridden by an option in slot-scoring rules. Also, a slot-scoring rule can change the basic formula applied, in a way that will not be described here.</Paragraph>
    <Paragraph position="39"> The combination of SlotPref and CloseAttach is closely related to preference rules discussed in \[19\].</Paragraph>
    <Paragraph position="40"> PhrasePref Some rules for this component are coded in the shell and have to do with preferences for the feature structure of the analysis of a complete input phrase, for example preference of finite clauses over noun phrases (except in certain environments).</Paragraph>
    <Paragraph position="41"> Phrase-scoring rules in XSG contribute to PhrasePref, and are of a form similar to slot-scoring rules- without mentioning a slot: +E ~ Body.</Paragraph>
    <Paragraph position="42"> The real number E is added to the total score whenever a phrase satisfying Body fills any slot, or is used as a conjunct in a coordinate structure, or is taken as a top-level analysis. A sample phrase-scoring rule in ESG is -I-I ~-- f(noun( ,, ,, * ) )&amp;-~Imod(ndet, , )&amp; t od(nadj, P)&amp;h odst(P, qu nt d ). This penalizes (by +1) a complete noun phrase that has no determiner but does have an adjective modifier which has some analysis with the feature quantadv. This rule penalizes for example the analysis of even Bill in which even is an adjective.</Paragraph>
    <Paragraph position="43">  WordSensePref All of the rules for this component are coded in the lexicon. An example is a lexical entry for the word man: man &lt; n( human&amp;male, nil) &lt; v( ev( 2 ), *, obj l ).</Paragraph>
    <Paragraph position="44"> The first lexical analysis element shows man as a noun with features human and male. The second analysis shows a verb word sense with a WordSensePref penalty of +2.</Paragraph>
    <Paragraph position="45"> These scores for word sense choices can also be coded conditionally on subject area codes, and there is an if-then-else formalism for expressing this.</Paragraph>
    <Paragraph position="46"> The WordSensePrefscore is added when an initial (one-word) phrase analysis is formed.</Paragraph>
    <Paragraph position="47"> HeadSlotFiller Following a method due largely to Ido Dagan \[2\], counts of head-slot-filler occurrences are obtained by parsing a corpus with ESG. Actually parts of speech are stored along with the head words of the higher and modifier phrases, so the counts are of quintuples: ( HWord, H POS, Slot, MWord, M POS).</Paragraph>
    <Paragraph position="48"> These counts are then used (with certain coefficients) to add a reward (a negative number) to the score each time a modifier is attached with a match to a stored quintuple.</Paragraph>
    <Paragraph position="49"> POSPref Using an idea of Ido Dagan and Herbert Leass, ESG corpus parsing is used to obtain counts of occurrences of pairs (Word, PartO f Speech).</Paragraph>
    <Paragraph position="50"> When an initial (one-word)phrase analysis is formed, and the word and its part of speech match an entry in the table just mentioned, then the count, with a certain negative coefficient, is added as the POSPrefcontribution to the phrase score. This is of course similar to WordSensePref, taken from the lexicon, and there is an overlap.</Paragraph>
    <Paragraph position="51"> LexSIotPref Rules for this component are coded in the lexicon. A slot apprearing in a slot frame in a lexicai entry can have an associated semantic type test on its filler. For example consider the following entry for give (not an actual entry for ESG): give &lt; v(obj . iobj : human).</Paragraph>
    <Paragraph position="52"> Here the iobj slot requires a filler with the semantic type human. (In general, any Boolean combination of type tests can be coded.) If this analysis is used, then aLexSlotPrefscore of -1 is added. As it is stated, this semantic type requirement is absolute. But if one writes give &lt; v( obj . iobj : pre f ( human) ).</Paragraph>
    <Paragraph position="53"> then the test is not an absolute requirement, but merely gives a score increment of -1 if it is satisfied. In both the absolute and the &amp;quot;fallsoft&amp;quot; forms of semantic type tests, the formalism allows one to specify arbitrary score increments.</Paragraph>
  </Section>
class="xml-element"></Paper>