The GE NLToolset: 
A Software Foundation for Intelligent Text Processing 
Paul S. Jacobs and Lisa F. Rau 
Artificial Intelligence Program 
CI'; Research and Development Center 
Schenectady, NY 12301 USA 
rau~crd.ge.com, psjacobsC~crd.ge.com 
Many obstacles stand in the way of computer pro- 
grams that could read and digest volumes of natu- 
ral language text. The foremost of these difficulties 
is the quantity and variety of knowledge about lan- 
guage and about the world that seems to be a pre- 
requisite for any substantial language understanding. 
In its most general form, the robust text processing 
problem remains insurmountable; yet practical ap- 
plications of text processing are realizable throngh 
a combination of knowledge representation and lan- 
guage analysis strategies. 
This project note describes the GE NLToo~s~,:T 
and its use in two text processing applications. In 
the first, dornain, the system selects and analyzes sto- 
ries about corporate mergers and acquisitions as they 
come across a real-time news feed. In the second do~ 
main, the program uses naval operations messages to 
fill a 10--field template. In both cases, users can ask 
natural language questions about, the contents of the 
texts, and the system responds with direct answers 
along with the original text. 
The GE NLTooLsET is a software foundation for 
text processing. The NL'I'OOLS~?'r derives from a 
research effort aimed at preserving the capabilities 
of naturM language text processing across domains. 
The program achieves this transportability by using 
a core knowledge base and lexicon that customizes 
easily to new applications, along with a flexible text 
processing strategy tolerant of gaps in the program's 
knowledge base. Developed over the last four years, 
it runs in real time on a SUN TM workstation in Com- 
mon Lisp under UNIX TM. It performs the following 
t asks: 
• The lexical analysis of the input character 
stream, including names, dates, numbers, a, nd 
eorttractions. 
• The separation of the raw news feed into story 
structures, with separate headline, byline and 
dateline designations. 
• A topic determination fbr each story, indicating 
whether it is about a corporate merger. 
• The natural language analysis of each selected 
story using an integration of two interpretation 
strategies--"bottom-up" linguistic analysis and 
"top-down" conceptual interpretation. 
o The storage and retrieval of conceptual represen- 
tations of the processed texts into and out of a 
knowledge base. 
The design of the NLTooLsET combines arti- 
ficial intelligence (AI) methods, especially natural 
language processing, knowledge representation, and 
information retrieval techniques, with more robust 
but superficial methods, such as lexical analysis and 
word-based text search. This approach provides the 
broad flmctionality of AI systems without sacrific- 
ing robnstness or processing speed. In fact, the 
system has a throughput for real text greater than 
any other text extraction system we have seen (e.g., 
\[Sondheimer, 1986; Sundheim, 1990\]), while provid- 
ing knowledge-based capabilities such as producing 
answers to English questions and identifying key con- 
ceptual roles in the text (such as the suitor, target, 
and per-.share price of a merger offer). The NL- 
TooLs~'r consists of roughly 50,000 lines of Common 
Lisp code. It was developed entirely on SUN work- 
stations. 
1 Technical Overview 
The NLTOoLSFT's design provides each system com- 
ponent with access to a rich hand-coded knowledge 
base, but each component applies the knowledge se- 
lectively, avoiding the computation that a complete 
analysis of each text would require. The architecture 
of the system allows for levels of language analysis, 
f¥om rough skimming \[Jacobs, 1990\] to in-depth con- 
ceptual i:nterpretation \[aacobs, 1987\]. 
A custom-built 10,000 word-root lexicon and con- 
cept hierarchy provides a rich source of lexical infor- 
mation. Entries are separated by their senses, and 
contain special context clues to help in the sense- 
disambiguation process. A morphological analyzer 
contains semantics for about 75 affixes, and can au- 
tomatically derive the meanings of inflected entries 
not separately represented in the lexicon. Domain- 
specific words and phrases are added to the lexicon 
by connecting them to higher-level concepts and cat- 
egories present in the system's core lexicon and con~ 
cept hierarchy. This is one aspect of the NLTOOLSET 
that makes it highly portable from one domain to an- 
other. 
The language analysis strategy used in the NL- 
TOOLSET combines full syntactic (bottom-up) pars- 
ing and conceptual expectation-driven (top-down) 
parsing. Four knowledge sources, including syntactic 
and semantic information and domain knowledge, in- 
teract in a flexible manner. This integration produces 
a more robust semantic analyzer that deals gracefully 
with gaps in lexieal and syntactic knowledge, trans- 
1 373 
.~m i~ :.'~:V& ~ ................ 
Article Number: 4 
Classlflcation: Takeover 
~.A. IIANNA CO. ACQUIRES 
~RU(~( PLASTICS CO° 
CLEVELAND -DJ- M.A. HANNA CO. SAID IT 
:OMPI.ETED ITS PREVIOUSLY REPORTED ACQUISITION OF 
~RUCK PLASTICS CO., A POLYMER RESINS DISTRIBUTOR 
9ASED NEAR CHICAGO, FOR UNDISCLOSED TERMS. 
BRUCK HAS ANNUAL REVEHDE OF ABOUT $I00 
~ILLION. 
-0- 9 28 AM EDT 04-03-89:"?; 
.... , , 
i~mm.#m+<,i'. II i III I - 
tcle Mo. : 20 
ar~%: Mayfair Super Harkets 
TA~ .... 
AH:icle Mo.: 37 
Tar~%: Major Realty 
Suitor: Stoneridge 
*~*n~TICLE 4 STORY REPRESENTATION ~* 
C-CORP-TAKEOVER 
R-SUBEVENT: VERB_COMPLETEI 
R-TARGET: 
C-BUSINESS-ORG 
R-CO-SALES: $ 100,000,000 
R-NAME: Bruck Plastics 
R-SUITOR: 
C-BUSINESS-0RG 
R-NAME: M A Hanna 
L9: Copper al~r N. ~, aay 131.80 dn .80 \]ly 124.50 da 1.50 platinu~ apr 526.00 
~0: I~rfair 9mrs lmyout p~posal 
+~1: Burger king nyse opening prices 
~+2: Mca Inc opening delayed- order ilbalance- last selsctronlcs outlook 
\]3: Poehl- bundesbank- in%erventlon 
!4: Ih, srnight 9 7- 8- 3- 4 "to~,- next g 7- 8- 3- 4 sloo%- next 9 7- 8- 3- 4 7 
25: 7 C a zhr vs op net 9 c 
215: ~d~ C8~?. za\]~ $ 9.75 ~ ~hr le~Jer ~m~ ~m~le%~ 
27: CORP. clasz b's last sale was 2 7- 8. 
+-- .,;,r, tn.m~m ............ |, 
~6\] ~ne /ho~e/~culpto~/u2/~ 
~upka/nlt2.0/Tooltool/~ 
~.4/tooltool ~f ... 
:63 16631 
Ray Text .~ 
"- -(3~ Tokenized Article "* 
JL,+tlcl. No. : 51 d\[,._.O~.d~J t ¢-+o wv. j ~ ~ ~ Ill .... '" 
Target: American Building Kaintenauce t /''~' ' ++ -----~-~ - --- It ~m ~esUo~ + 
.99 = ~ ~ ~ .... -~|l~++m- ...... __ _ -, ................ ~t~r the article number: 4 ~~ 
i+~+++~++:+~+++++++++~+~~++++i~+++~+~+++++++i+~++~++++++++++++~m+~.`~+++++~++++++~i+++i++i~+++~+++++++++++++++++~;+++P:++++~++++++++~+++++++++++~:~:++.~++++t+++~++~+++++++~.~'~+.~.~++~+++++i++++#~++++++~~~ 
Figure h Sclsoa in action 
ports easily to new domains, and fanilitates the ex- 
traction of information from texts \[Rau and Jacobs, 
1988\]. 
Two prototype systems (both to be demonstrated 
at Coling) illustrate some of the capabilities of the 
NLTooLSET. SCISOR (System for Conceptual In- 
formation Summarization, Organization, and Re- 
trieval) reads financial news stories from a news ser- 
vice, selects stories about mergers and acquisitions, 
extracts key pieces of information from those sto- 
ries, and answers English questions about this infor- 
mation. MUCK-II (a demonstration from a mes- 
sage understanding conference in 1989 \[Sundheim, 
1990\]) shows some of the same capabilities, includ- 
ing database generation, question answering, and au- 
tomatic alert, applied to a set of naval messages 
(OPREP-3). Both systems process texts at a rate 
of hundreds of paragraphs per hour. The customiza- 
tion of the NLTOoLSET to the MUCK-II applica- 
tion, porting from the domain of corporate takeovers 
to naval operations, required only several weeks. 
2 SCISOR 
SCISOR is a customization of the NLTOOLSET to 
the domain of news stories about mergers and acqui- 
sitions. The program analyzes stories a.s they come 
across a live news feed, selecting the takeover stories 
and applying a combination of top-down and bottom- 
up language analysis to identify conceptual roles in 
the stories. The result of this analysis is a single rep- 
resentation of each story that the program adds to 
a central knowledge base. The conceptual retrieval 
component accesses information from this knowledge 
base by analyzing English questions in the same man- 
ner and matching the questions to the story represen- 
tations stored in the knowledge base. 
ScBoR provides the user with information in mul- 
tiple forms. Users can browse the headlines and the 
original texts. A "hot window" continuously displays 
the target, suitor, and price of the latest takeover 
stories, and flashes when a new takeover story comes 
across the wire. For more general information needs, 
an "ask question" window allows the user to type in 
simple English questions (e. g., "What was offered 
for Polaroid?") as well as query fragments (e. g., 
374 2 
"acquisitions by Shamrock"). 
Figure 1 shows a SUN screen during the opera- 
tion of SClsoR. The "Master Control" window in 
the lower right allows the user to open or access the 
variou,'~ features of the system. The "}Ieadlincs" and 
"Display Control" in the lower center show the head- 
lines of all stories (with headlines of takeover stories 
in bold) and guide the selection of texts for browsing. 
The "Hot Window", or alert feature, is at the lower 
left, alerting users the instant a new, potentially rel- 
evant article comes across the news wire. The "Raw 
Text" and "Trump Representation" windows at the 
top display each selected story, showing key portions 
of text in boldface with a summary of the language 
analysis in the upper right. 
More details on the system design and operation of 
SCISOR can be found in \[Jacobs and Rau, 1990\]. 
3 Performance Evalution 
Performance evaluation of natural language systems 
is a new problem, although the evaluation methods 
can adopt some of the techniques of traditional infor- 
mation retrieval (IR) systelns. It would be difficult 
and probably futile to perform a controlled study of 
the NLTOOLSET against a traditional IR system, for 
two reasons: (1) traditional IR systems are tested 
on ~bitrary, unconstrained texts, while natural lan- 
guage systems still work only in constrained domains; 
(2) the NLTOOLSET performs many tasks other than 
document retrieval, such as extracting information 
from stories and directly answering users' questions. 
Evaluation problems of the entire system stem from 
the unique functionality of the NLTOoLS~;T system. 
Document retrieval systems, even sophisticated ones 
like RuBRIc\[Tong et al., 1986\], do not extract fea- 
tures from from the documents they retrieve; thus it 
is impossible to compare them to NI,TooLsET. tfow- 
ever, we have performed some tests that do measure 
the NLTooLS~T's accuracy in specific tasks. 
The government-sponsored MUCK-II evaluation 
is, to our knowledge, the most meaningful test of nat- 
ural language text processing, but the participants in 
the MUCK-II evaluation agreed not to release the 
specific results of the experiment. Itowever, we will 
try to summarize the status of performance evalua- 
tion in general terms. Evaluation of content-based 
text processing systems like SclsoP~ is not nearly as 
established as evaluation methods in information re- 
trieval. There are many tasks to be tested in this 
emerging type of system, including accuracy of ques- 
tion answering, helpfulness of alerts, and coverage of 
structured information (such as target and suitor). 
No mature methods exist for testing any of these 
tasks. 
In spite of the problems with evaluating this sort 
of system, we would like to be informative about 
how our program performs. As a rule, it can extract 
key features from large sets of constrained texts with 
80-90% (combined recall and precision) accuracy. It 
can achieve better results (and has) with more con- 
strained texts, but would also produce almost nothing 
useful, say, in reading the entire Wall Street Jour- 
nal. It is realistic to expect 90% accuracy for certain 
useflfl, carefiflly-constructed tasks, and unrealistic to 
expect much higher than this 1. Many ditficulties in 
reading texts appear when trying to achieve better 
results, but the most common limitation seems to be 
the degree of real inference required for understand- 
ing. In spite of its fairly sophisticated methods for 
combining linguistic and world knowledge, the NL- 
TOOLSE'r really has very little of the latter. 
In a recent test of ScISOR, the program analyzed 
one day's worth of stories directly from the newswire 
source. Of the 729 stories, the filter achieved slightly 
over 90% averaged recall and precision in its deter- 
mination of which stories were about mergers and 
acquisitions (69 in all). Sclso~t correctly identified 
the target and suitor in 90% of all the stories. When 
dollar-per-share amounts of offers were present in the 
stories, Sclso~t extracted this quantity correctly 79% 
of the time, and the total value of the offer 82% of 
the time. 

References 

\[DeJong, 1979\] Gerald DeJong. Prediction and sub- 
stantiation: A new approach to natural language 
processing. Cognitive Science, 3(3):251---273, 1979. 

\[Jacobs and Ran, 1990\] Paul Jacobs and Lisa llau. 
SCISOR: A system tbr extracting information from 
on-line news. Communications of the Associa- 
tion for Computing Machinery, 35, (in. $ubm.is- 
sion) 1990. 

\[Jacobs, 1987\] Paul S. Jacobs. A knowledge frame- 
work tbr natural language analysis. In Proceedings 
of the Tenth International Joint Conference on Ar- 
tificial Intelligence, Milan, Italy, 1987. 

\[Jacobs, 1990\] P. Jacobs. To parse or not to parse: 
Relation~driven text skimming. In Proceedings of 
the Thirteenth Inter'national ConfereT~ce on Com- 
putational Linguistics, IIelsinki, Finland, 1990. 

\[Rau and Jacobs, 1988\] Lisa F. Rau and Paul S. Ja- 
cobs. Integrating top-down and bottom-up strate- 
gies in a text processing system. In Proceedings of 
Second Conference on Applied Natural Language 
Processing, pages 129-135, Morristown, NJ, Feb 
1988. ACL. 

\[Sondheimer, 1986\] N Sondheimer. Proceedings of 
DARPA's 1986 strategic computing natural lan- 
guage processing workshop. Technical Report 
ISI/SR-86-172, University of Southern California, 
ISI, 1986. 

\[Sundheim, 1990\] Beth Sundheim. Second message 
understanding conference (MUCK-II) test report. 
Technical Report 1328, Naval Ocean Systems Cen- 
ter, San Diego, CA, 1990. 

\[Tong et al., 1986\] Richard M. Tong, L. A. Appel- 
baum, V. N. Askman, and J. F. Cunningham. 
RUBRIC iII: An object-oriented expert system for 
information retrieval. In Proceedings of the 2nd An- 
nual IEEE Symposium on Expert Systems in Gov- 
e~nrnent, W~hington, DC., October 1986. IEEE 
Computer Society Press. 

1The I"l{UMP\[DeJong, 1979\] program, for comparison 
purposes, achieved 38% accuracy in one test on newswire 
stories. 
