<?xml version="1.0" standalone="yes"?>
<Paper uid="M93-1022">
  <Title>DEVELOPMENT EFFORT S NUBA was implemented in C++ . The software consists of three parts :</Title>
  <Section position="1" start_page="0" end_page="0" type="metho">
    <SectionTitle>
INTRODUCTIO N
</SectionTitle>
    <Paragraph position="0"> Abduction is the inference to the best explanation . Many tasks in natural language understanding such as word-sense disambiguity [1], local pragmatics [4], metaphor interpretation [3], and plan recognition [5, 8] , can be viewed as abduction.</Paragraph>
    <Paragraph position="1"> NUBA (Natural-language Understanding By Abduction) is a natural language understanding system , where syntactic, semantic, and discourse analysis are performed by abductive inference . The task of understanding is viewed as finding the structural relationships between unstructured inputs . That is, to understand is to seek the best explanation of how the inputs are coherently related . From this abductive perspective , the goal of the parser is to explain how the words in a sentence are structured according to syntactic relationships; the goal of the semantic interpreter is to find the semantic relationships among the content word s in a sentence ; the goal discourse analyzer is to show how the events mentioned in the sentences fit togethe r to form a coherent plan .</Paragraph>
    <Paragraph position="2"> Although the abductive formulation of natural language understanding tasks results in significant simpli fications [4], the computational complexity of abductive inference presents a serious problem . Our solution to this problem is obvious abduction [6], a model of abductive inference that covers the kinds of abductiv e inferences people perform without apparent effort, such as parsing, plan recognition, and diagnosis . Obvious abduction uses a network to represent the domain knowledge . Observations correspond to nodes in th e network annotated with a set of attribute-value pairs. An explanation of the observations is a generalize d subtree of the network that connects all the observations . This connection is a coherent set of relationships between the observations, therefore, explains how they are related .</Paragraph>
  </Section>
  <Section position="2" start_page="0" end_page="0" type="metho">
    <SectionTitle>
SUMMARY OF TEST RESULTS
</SectionTitle>
    <Paragraph position="0"> Considering that NUBA is a new participant in MUC and that several important modules of NUBA hav e not yet been implemented, NUBA's test results are very good . Table 1 shows a summary of the scores .</Paragraph>
  </Section>
  <Section position="3" start_page="0" end_page="0" type="metho">
    <SectionTitle>
SLOT REC PRE UND OVG ERR UND OVG SUB
ALL OBJECTS
MATCHED ONLY
TEXT FILTERING
</SectionTitle>
    <Paragraph position="0"/>
  </Section>
  <Section position="4" start_page="0" end_page="0" type="metho">
    <SectionTitle>
SYSTEM ARCHITECTUR E
</SectionTitle>
    <Paragraph position="0"> Figure 1 shows the NUBA system architecture . NUBA has the following innovative aspects, compared with the generic information extraction system [2] .</Paragraph>
    <Paragraph position="1"> * The lexical analyzer creates a set, instead of a sequence, of lexical items . This means that the surface strings of the lexical items may overlap .</Paragraph>
    <Paragraph position="2">  little, if anything at all, in common. In NUBA, they share the same message passing algorithm for obvious abduction . They differ only in the contents of messages and the constraints on messag e combination and propagation .</Paragraph>
  </Section>
  <Section position="5" start_page="0" end_page="264" type="metho">
    <SectionTitle>
LEXICAL ANALYSIS
</SectionTitle>
    <Paragraph position="0"> The lexical analyzer recognizes the sentence boundaries and creates a set of lexical items for each sentence .</Paragraph>
    <Paragraph position="1"> A lexical item is a pair : &lt;surface-string, attribute-vector&gt;, where surface-string is an interval [i,j] denoting the i'th to j'th word in the sentence ;  attribute-vector is a list of attribute-value pairs . The attributes may either be syntactic, e .g., +plu, (per 3)., or semantic, e.g., (film silicon), (layering CVD) .</Paragraph>
    <Paragraph position="2"> In case of lexical ambiguity, multiple items are created for the same word or phrase . The surface strings of different items may even overlap .</Paragraph>
    <Paragraph position="3"> A LEX program is used for message zoning and sentence boundary recognition . For each sentence, the lexical analyzer then  1. maps the words and phrases in a sentence into a set of lexical items by looking up a lexicon . 2. applies a set of lexical rules to the lexical items .</Paragraph>
    <Paragraph position="4">  These two steps are discussed in the next two subsections .</Paragraph>
  </Section>
  <Section position="6" start_page="264" end_page="265" type="metho">
    <SectionTitle>
LEXICON
</SectionTitle>
    <Paragraph position="0"> The purpose of lexicons is to map words into their semantic and/or syntactic representations . NUBA 's lexicon consists of two files : one holds the entries, the other contains a hash index into the first file. None of these files are loaded into the memory . When changes are made to the lexicon, the hash index file has to be rebuilt. In our experiment, where the lexicon contains 90K entries, the average time to retrieve an entry i s 0.002 second.</Paragraph>
    <Paragraph position="1"> Lexical entries are written in LISP-like format . It contains a key, which may consist of more than on e word, and a list of functions . The functions return either a meaning of the key or a list of phrases for whic h the key is the head word.</Paragraph>
    <Paragraph position="2"> The format of an lexical entry is as follows :  The function meaning returns a Meaning object. The first argument is the class of the meaning object , which is a subclass of Meaning. The arguments following the class are passed to the initializer for the class . The first entry means that the word aluminum can be a type of bonding, or a type of film, or the head wor d in phrases Aluminum Co of America, aluminum copper, and aluminum silicon.</Paragraph>
    <Paragraph position="3"> The second entry is a phrasal entry (chemical beam epitaxy). The word epitaxy is said to be th e head of the phrase . Generally speaking, the head word of a phrase should be the least frequent word in th e phrase or one that may undergo morphological changes . When the head word is found to be present in a sentence, the lexicon then check its neighboring words in the sentence to see whether the phrase is presen t or not. If it is, one or more lexical items for the phrase are created .</Paragraph>
  </Section>
  <Section position="7" start_page="265" end_page="265" type="metho">
    <SectionTitle>
LEXICAL RULES
</SectionTitle>
    <Paragraph position="0"> Once the lexical items have been obtained by looking upt the lexicon, a set of lexical rules is applied to them .</Paragraph>
    <Paragraph position="1"> Each of the following task is performaed by a lexical rule : Corporate name recognition: An entity name will be recognized if it appears in the lexicon . Otherwise , if it is followed by a corporate designator, it may be recognized by this rule . When a lexical item has +corpdesig but the name attribute is undefined, the word is a corporate designator . The rule then searches for the sequence of capitalized words before this word and interprets the sequence as the name of an entity.</Paragraph>
    <Paragraph position="2"> Irrelevant sentence filtering : When a non-1C word or phrase, such as &amp;quot;printed circuit board,&amp;quot; is foun d in a sentence, the sentence is assumed to be irrelevant and all the lexical items are removed .</Paragraph>
    <Paragraph position="3"> Negation handling: Since the current implementation does not include a parser, the scope of negation operator is simply assumed to be from its position to the end of the sentence . When a negation word (e.g., not, no, except) is encountered, all the lexical items following the word are removed .</Paragraph>
    <Paragraph position="4"> City name recognition: The lexicon contains all the country names and the provinces for several countrie s that are most frequently mentioned in the training corpus . When a country or province name is preceded by a sequence of capitalized unknown words, the sequence is assumed to be a city name .</Paragraph>
    <Paragraph position="5"> Determination of location of entities : Locations of entities are determined if they appear either befor e or after the entity in the text .</Paragraph>
  </Section>
  <Section position="8" start_page="265" end_page="265" type="metho">
    <SectionTitle>
PRINCIPLE-BASED PARSIN G
</SectionTitle>
    <Paragraph position="0"> In [7], the author presented an efficient, principle-based parser . The parser encodes the Government and Binding (GB) theory in a network . The nodes in the network represent grammatical categories, such as NP, VP, etc . The links in the network represent dominance relationships between the categories . The GB principles are represented as constraints attached to nodes and links in the network . The lexical items are mapped into nodes in the network, annotated with attribute values . The algorithm for obvious abduction i s used to find connections between the words in the network, which are consistent with principles in the G B Theory. The connections explain how the words in the sentence relate to one another in terms of syntacti c relationships and can serve as the parse trees of the sentence .</Paragraph>
    <Paragraph position="1"> The parse has been implemented and preliminarily tested . However, due to the shortage of time and people, we were not able to integrate the parser with the rest of the system before MUC-5 formal testing .</Paragraph>
  </Section>
  <Section position="9" start_page="265" end_page="266" type="metho">
    <SectionTitle>
SYNTAX-CONSTRAINED SEMANTIC INTERPRETATIO N
</SectionTitle>
    <Paragraph position="0"> In NUBA, the domain knowledge is represented by a semantic network (Figure 2) . Semantic interpretation is viewed as the process of finding the best explanation of how the content words in the sentence are relate d to one another in terms of semantic relationships in the network .</Paragraph>
    <Paragraph position="1"> A lexical item corresponds to a node in the semantic network, annotated with a set of attribute values .</Paragraph>
    <Paragraph position="2"> The goal of semantic interpretation is to find a generalized subtree of the network that connects the lexica l items. A tree is a generalized subtree of the network if the nodes in the tree are labeled with the nodes in the network and every directed path in the tree is also a directed path in the network . We call the connection a scenario. Such a scenario explains the lexical items because a description of the scenario may mention th e surface strings in the lexical items. The best explanation is one that explains the largest number of lexica l items with the minimum number of links .</Paragraph>
    <Paragraph position="3"> The algorithm for finding the best explanation is a message passing algorithm . A message is a pair tha t represents an explanation of a subset of the lexical items : ([b, e], av)  where [b, e] is an integer interval representing the span of the lexical items it explains and av is an attribute value vector representing the properties of the explanation . Upon receiving a message, a node attempts to combine it with other messages already received by the node to form a new message. That is, the node combines several smaller explanations into a larger one. Two messages ml = ([bl , el], awl) and m2 = ([b2 , e 2 ], av2) can only be combined if their spans do not overlap, i .e., el &lt; 62 , and their attribute values ar e unifiable . The result of the unification is m = ([b 1 , e 2], unify(avi , av2 )) .</Paragraph>
    <Paragraph position="4"> Each node has a completion predicate. If the attribute values of a message satisfy the completion predicate, the message is sent further to other messages . Otherwise, the message waits to be combined wit h other messages at the node .</Paragraph>
    <Paragraph position="5"> Filters can be attached to the links in the semantic network . A filter is an attribute value vector . A message can only pass through the filter if its attribute values are unifiable with those of the filter . For example, the link from lithography to equipment has a filter (muctype lithography). This means that if an equipment has a muctype attribute but its value is not lithography, then the equipment cannot b e involved in a lithography process.</Paragraph>
    <Paragraph position="6"> When the message passing process stops, we can find the best explanation by tracing the origins of th e messages that explain the largest number of lexical items . The number of lexical items that are explaine d by a message is the value of the count attribute in the message.</Paragraph>
    <Paragraph position="7"> Semantic disambiguity is achieved as a side effect of the search for the best explanation . The explanatio n tree connects at most one of the senses of a word or phrase with the other words in the sentence . Since we search the tree with minimum total length, the selection of the sense is globally optimal as opposed t o locally optimal in many other methods, such as marker passing in semantic networks .</Paragraph>
    <Paragraph position="8"> Consider an example sentence : Applied Materials , Inc . today announced new aluminum etch capability with its single-wafer , multi-chamber Precision 5000 Etch system The following lexical items are created by the lexical analyzer. The semantic interpreter then found a  generalized subtree of the semantic network that connects the lexical items : 1. [0,3] Applied Materials , Inc. ((muctype name) +cap +corpdesig (name &amp;quot;Applied Materials&amp;quot; ) (deaig &amp;quot;INC&amp;quot;) (mucnode &amp;quot;entity&amp;quot;) ) 2. [7,7] aluminum ((mucnode &amp;quot;film&amp;quot;) (film ALUMINUM) ) 3. [7,7] aluminum ((mucnode &amp;quot;bonding&amp;quot;) (bonding ALUMINUM) ) 4. [8,8] etch (+etch (mucnode &amp;quot;etching&amp;quot;) ) 5. [8,8] etch ((muctype etching) +etch (mucnode &amp;quot;equipment&amp;quot;) (equipment ETCHING-SYSTEM) ) 267 6. [15,16] Precision 5000 (+equip (model &amp;quot;Precision 5000&amp;quot;) (mucnode &amp;quot;equipment&amp;quot;)) 7. [17,17] Etch (+etch (mucnode &amp;quot;etching&amp;quot;) ) 8. [17,17] Etch ((muctype etching) +etch (mucnode &amp;quot;equipment&amp;quot;) (equipment ETCHING-SYSTEM) ) 9. [18,18] system (-cap +equip +postmod (mucnode &amp;quot;equipment&amp;quot;))</Paragraph>
  </Section>
  <Section position="10" start_page="266" end_page="268" type="metho">
    <SectionTitle>
INTEGRATION WITH SYNTA X
</SectionTitle>
    <Paragraph position="0"> Previous approaches to semantic interpretation can be classified as either one of the following : Syntax-guided : Semantic structures are derived from a parse tree or parse tree fragments . The disadvantage of this is that the semantic analysis is critically dependent upon the output of syntactic analysi s and syntactic ambiguities have to be resolved before semantic analysis .</Paragraph>
    <Paragraph position="1"> Frame-guided : Semantic interpretation is driven by instantiation of frames that are triggered by keywords .</Paragraph>
    <Paragraph position="2"> The problem with this approach is that there is no principled method for controlling the interaction o f multiple frames that may be triggered by the same word or the same set of words . Further more, this approach often results in complex frame definitions that are difficult to port to another domain .</Paragraph>
    <Paragraph position="3"> Semantic interpretation in NUBA is syntax-constrained in the sense that the semantic structure must be consistent with syntactic structure. The notion of structural-consistency between semantic and syntacti c structures is similar to the structural-consistency between parse trees [9] .</Paragraph>
    <Paragraph position="4"> Definition 6.1 (Span) . An integer interval [i, j] is said to be a span of a sentence if there exists a parse tree and a node n in the parse tree such that the i 'th to j'th word in a sentence is dominated exactly by a consecutive subtrees of n .</Paragraph>
    <Paragraph position="5"> The difference between the notion of span here and [9] is that the latter requires that a span consists of all the words that are dominated by a single non-terminal symbol . Figure 3 shows the spans in a parse tree .</Paragraph>
    <Paragraph position="6"> The spans in a parse forest is the unions of the spans in the trees in the forest.</Paragraph>
    <Paragraph position="7"> Semantic interpretation in NUBA is based on : Definition 6 .2 (Structural-consistency Hypothesis) . The spans in semantic dependency structure are a subset of the spans in the parse forest .</Paragraph>
    <Paragraph position="8">  From a packed shared parse forest, we can derive all the allowable spans in a sentence. During the message passing process, the messages are combined by a node only if the span of their combination is an allowable span according to the parse forest .</Paragraph>
    <Paragraph position="9"> If the parse forest is not available, the parse tree is assumed to be a flat structure, where the category S (sentence) immediately dominates all the word (Figure 4) . This means that any interval [i, j] is an alloabl e span. The parse tree impose no constraints on the semantic structure. This is what we did in our official system .</Paragraph>
    <Paragraph position="10"> Note that even though our official system do not make use of the parse forest, the connection trees obtained by the semantic analyzer are not arbitrary . They must be convex with respect to the sequence of words in the sentence.</Paragraph>
    <Paragraph position="11"> Definition 6.3 (Convexity) . A tree connecting a sequence of elements is convex with respect to the sequence iff for any two elements wi , wi (i &lt; j) in the sequence, any node in the tree that dominates both wi and w1 must also dominate all the element between w i and w1 .</Paragraph>
    <Paragraph position="12"> Figure 5 shows examples of a convex and a non-convex tree . Although the syntax of different languages may be very different, they all seem to satisfy the convexity constraint . Therefore, without any help of a parser , the semantic analyzer in NUBA is still able to take advantage of a common denominator of the syntacti c constraints in different languages.</Paragraph>
  </Section>
  <Section position="11" start_page="268" end_page="269" type="metho">
    <SectionTitle>
DISCOURSE ANALYSIS
</SectionTitle>
    <Paragraph position="0"> The discourse module in our official system is a striped-down-to-the-bare-bone version . The only function implemented is the unification of scenarios . The system maintains a list of scenarios . Whenever possible, a</Paragraph>
    <Paragraph position="2"> newly generated scenario is merged with the scenarios in the list . If it is not unifiable with any of them, the new scenario is inserted into the list.</Paragraph>
  </Section>
  <Section position="12" start_page="269" end_page="269" type="metho">
    <SectionTitle>
TEMPLATE GENERATIO N
</SectionTitle>
    <Paragraph position="0"> The structure of the semantic network, hence the structure of the scenarios, is very similar to templat e structures . Once the scenarios have been identified, template generation is quite straightfowrard . One or more MICROELECTRONICS-CAPABILITY templates will be generated for each scenario . The roles of the entities involved in the scenario are determined by the attribute values of the me-cap node in the scenario .</Paragraph>
    <Paragraph position="1"> For example, if +distribute is present, then the entity fills the distributor role . When there are multipl e entities, whether they are fillers of the same role or separate roles is determined by the joint attribute in the me-cap node .</Paragraph>
  </Section>
  <Section position="13" start_page="269" end_page="272" type="metho">
    <SectionTitle>
ANALYSIS OF THE WALKTHROUGH EXAMPL E
</SectionTitle>
    <Paragraph position="0"> We now show how the the walkthrough example is processed by our official system . The first sentence of the walkthrough example is : In the second quarter of 1991 , Nikon Corp. &lt;left&gt; 7731 &lt;right&gt; plans to market the &lt;q&gt; NSR-1755EX8A , &lt;/q&gt; a new stepper intended for use in the production of 64-Mbi t DRAMs The following lexical items have been identified by the lexical analyzer :  1. [7,8] Nikon Corp. ((muctype name) +cap +corpdesig (name &amp;quot;Nikon&amp;quot;) (desig &amp;quot;CORP&amp;quot;) (mucnode &amp;quot;entity&amp;quot;)) 2. [14,14] market (+manufacture +distribute (mucnode &amp;quot;me-cap&amp;quot;) (count 0) ) 3. [17,17] NSR-1755EX8A (+equip (model &amp;quot;NSR-1755EX8A&amp;quot;) (mucnode &amp;quot;equipment&amp;quot;) ) 4. [22,22] stepper ((muctype lithography) +equip +lith (mucnode &amp;quot;equipment&amp;quot;) (equipment STEPPER)) 5. [28,28] production (+manufacture +distribute (mucnode &amp;quot;me-cap&amp;quot;) (count 0)) 6. [31,31] DRAMs ((muctype device) (device DRAM) )  The only relevant lexical item that is not identified by NUBA is the size of the device 64-Mbit, this is because our official system does not attempt to fill the device-size slot. These lexical items are sent to the nodes named in their mucnode attribute as initial messages . These messages will initiate a message passing process. When all the messages have been sent, NUBA finds the message at me-cap node with maximum count value. It then traces the origins of the message . The paths traversed during the trace form the following subtree of the semantic network, that connects the lexica l items:  equipment stepper ((muctype lithography) +equip (mucnode &amp;quot;equipment&amp;quot; ) +lith (count 1) (equipment STEPPER)) device DRAMs ((muctype device) (count 1) (device DRAM) ) This tree represents a scenario with one microelectronic capability. The entity involved is Nikon Corp. The process is lithography for making DRAM, where a stepper with model name NSR-1755EX8A is used as the equipment . Other than missing the device-size, all the information contained in this scenario is correct . The scenario is then inserted into the list of scenarios . NUBA proceeds to analyses the next sentence : The stepper will use an 248-nm excimer laser as a light source and will have a resolution of 0.45 micron , compared to the 0 .5 micron of the company &lt;g&gt; latest stepper The word compared is treated similarly as negation words, all the lexical entries after the word are delete d by a lexical rule . The phrase that begins with compared is actually a reference to the second microelectronic capability . The slot granularity is also ignored in our official system . Therefore, the lexical item corresponding to 0.45 micro is not identified by the lexical analyzer. The remaining lexical items are :  1. [1,1] stepper ((muctype lithography) +equip +lith (mucnode &amp;quot;equipment&amp;quot;) (equipment STEPPER)) 2. [6,7] excimer laser ((muctype lithography) +equip (mucnode &amp;quot;equipment&amp;quot;) (lithography LASER) (equipment RADIATION-SOURCE) )  The fact that one type of equipment can be part of another equipment is not represented in the semantic net work . Therefore, the program thinks that the stepper and the excimer laser are two separate equipments . As a result, NUBA fails to infer that the type of the previous lithography process is LASER and generates a spurious microelectronic capability template. Since no entity is mentioned in the sentence, NUBA assumes it to be the latest reference to an entity, Nikon Corp., and generates the following scenario : me-cap ((count 2)) process ((count 2) ) lithography ((muctype lithography) +lith (count 1) (equipment STEPPER) ) equipment stepper ((muctype lithography) +equip +lit h  This scenario means that the entity Nikon Corp. has two lithography processes . A stepper is used as equipment in one and a excimer laser is used as radiation source in the other. This scenario is unified with the previous scenario . The entity and the first lithography process is identical to that of the previous scenario . Therefore, only the second lithography process is added. The result of the unification is new scenario which replaces the previous one in the list of scenarios : me-cap (+manufacture +distribute (mucnode &amp;quot;me-cap&amp;quot;) (count 6) ) me-cap market (+manufacture +distribute (mucnode &amp;quot;me-cap&amp;quot;) (count 0) ) entities ((count 1)) entity Nikon Corp . ((muctype name) +cap +corpdesig (name &amp;quot;Nikon&amp;quot;) (desig &amp;quot;CORP&amp;quot;) (mucnode &amp;quot;entity&amp;quot;) (count 1) ) process ((count 3)) lithography ((muctype lithography) +lith (count 4) (equipment STEPPER) ) equipment ((muctype lithography) +equip +lith (model &amp;quot;NSR-1755EX8A&amp;quot;)  The third sentence is : Nikon will price the excimer laser stepper at 300-350 million yen , and the compan y expects to sell 50 systems during the initial year of marketing The lexical items are:  1. [0,0] Nikon ((muctype name) +cap (name &amp;quot;Nikon&amp;quot;) (mucnode &amp;quot;entity&amp;quot;) ) 2. [0,0] Nikon ((muctype name) +cap +corpdesig (name &amp;quot;Nikon&amp;quot;) (desig &amp;quot;CORP&amp;quot;) (mucnode &amp;quot;entity&amp;quot;) ) 3. [4,5] excimer laser ((muctype lithography) +equip (mucnode &amp;quot;equipment&amp;quot;) (lithography LASER ) (equipment RADIATION-SOURCE) ) 4. [6,6] stepper ((muctype lithography) +equip +lith (mucnode &amp;quot;equipment&amp;quot;) (equipment STEPPER) ) 5. [14,14] company ((muctype name) -cap (name &amp;quot;Nikon&amp;quot;) (mucnode &amp;quot;entity&amp;quot;)) 6. [17,17] sell (+manufacture +distribute (mucnode &amp;quot;me-cap&amp;quot;) (count 0) ) 7. [19,19] systems (-cap +equip +postmod (mucnode &amp;quot;equipment&amp;quot;)) 8. [25,25] marketing (+manufacture +distribute (mucnode &amp;quot;me-cap&amp;quot;) (count 0) )  There are two lexical items for the word Nikon due to a bug in a program . However, this bug does not make any difference in the analysis result of this sentence . The scenario generated for the third sentence is :  equipment systems (+equip +postmod (mucnode &amp;quot;equipment&amp;quot;) (count 1) ) 'Fhe information in this scenario is properly contained in the previous scenario obtained from the first tw o sentences . Therefore, there is no change in the list of scenarios .</Paragraph>
    <Paragraph position="1"> After analysesing all the sentences, there is one scenario in the list :  The template generator then fills in a set of templates according to the scenario . The scores of the templat e filling is:</Paragraph>
  </Section>
  <Section position="14" start_page="272" end_page="272" type="metho">
    <SectionTitle>
DEVELOPMENT EFFORT S
</SectionTitle>
    <Paragraph position="0"> NUBA was implemented in C++ . The software consists of three parts : Utilities: such as generic list, LISP-like command interpreter, bit vectors, attribute value vectors, lexicons , graphs, hash tables, binary searched files etc. (16,000 lines).</Paragraph>
  </Section>
  <Section position="15" start_page="272" end_page="273" type="metho">
    <SectionTitle>
TRAINING
</SectionTitle>
    <Paragraph position="0"> We did not use any automated training. The bulk of training time was spent on adding and modifyin g the lexicon entries . We used all of the 1000 training texts . However, only the articles with high error rat e were examined, which means that we spent a lot of time improving the text-filtering scores . This turne d out to be quite futile. Although we managed to improve the text-filtering score for the training articles, th e modifications we made did not generalize well to the testing articles .</Paragraph>
  </Section>
  <Section position="16" start_page="273" end_page="273" type="metho">
    <SectionTitle>
SOURCES OF KNOWLEDG E
</SectionTitle>
    <Paragraph position="0"> The semantic network is hand crafted, as it is fairly small. The lexical entries came from several sources .</Paragraph>
    <Paragraph position="1"> The model names and the company names were obtained from the key templates . NUBA can only recognize model names that appeared in the training data . However, the company names that do not appear in the training data can also be recognized if they are followed by a corporate designator . The country names an d province names are from the English gazetteer . Other entries in the lexicon are created manually. Most of them are based on the document for template filling rules .</Paragraph>
  </Section>
class="xml-element"></Paper>