THE MESSAGE UNDERSTANDING CONFERENCES 
Beth M. Sundheim 
Naval Command, Control, and Ocean Surveillance Center 
RDT&E Division (NRaD) 
Decision Support and AI Technology Branch 
San Diego, CA 92152-7420 
sundheim@nosc.mil 
The Message Understanding Conferences (MUCs) provide 
a forum for assessing and discussing progress in the field of 
natural language processing. Each conference is preceded 
by a formal evaluation of text analysis systems that have 
been developed to perform a shared task, as designed by the 
Government in consultation with evaluation participants 
from the research community. 
The evaluations and conferences bring research groups that 
are supported by major ARPA contracts together with other 
research groups, who are attracted by the opportunity to 
work on a well-defined task with Government-furnished 
materials, to meet with their peers, and to attract attention 
to their research and development program. Proceezlings of 
the last three conferences have been published in the open 
literature; two of those conferences were held in the period 
covered by Tipster Phase I \[1,2\]. 
A wide variety of system architectures, processing tech- 
niques, and tools have been tried, tested, and refined in the 
context of the evaluations, spurring progress in the robust 
processing of naturally, occurring text. To a large extent, it 
was the promise shown by the technology in the period 
between 1987 and 1991 that led the Government to include 
information extraction technology in the Tipster program. 
The task orientation and evaluation format for MUC were 
adopted by the TIPSTER prokram. 
Over the years since the first MUC in 1987, the text pro- 
cessing requirements of the evaluation have evolved from 
analyzing a small number of paragraph-length naval mes- 
sage narratives in a very restricted subject area (MUC-1 in 
1987 and MUC-2 in 1989) to analyzing a large number of 
full-length news articles in much broader areas. MUC-3 
(1991) and MUC-4 (1992) required processing of English 
newswire articles about Latin American terrorism. The 
MUC-5 evaluation, which ended in July. 1993, was based 
on the 'Iipster articles in ~n~lish and Japanese on joint ven- 
tures and microelectrouics. 
The application which serves as the basis for evaluating the 
text analysis systems is primarily information extraction, 
i.e., the identification and formatting of information con- 
tained in the news articles. (A secondary task is document 
detection, since some articles contain no pertinent iaforma- 
tioga whatsoever.) The MUC-5 application tasks were those 
that had been defined for the Tipster program. The systems 
developed by the Tipster-funded contractors (BBN, GEl 
CMU/NMSU/Brandeis, UMass/Hughes) were evaluated 
on the basis of their performance in English in both the 
joint ventures and microelectronics domains; all but the 
UMass/Hughes system were also evaluated in Japanese in 
both domains, 

References

\[1\] Proceedings of the Fourth Message Understanding 
Conference (MUC-4). June 1992. Morgan Kaufmann Pub- 
lishers. 

\[2\] Proceedings of the Fifth Message Understanding Con- 
ference (MUC-5), August 1993, Morgan Kanfmann Pub- 
lishers. 
