Revisions that Improve Cohesion in Multi-document Summaries:  
A Preliminary Study 
Jahna C. Otterbacher, Dragomir R. Radev and Airong Luo 
School of Information 
University of Michigan 
Ann Arbor, MI 48109-1092 
{clear, radev, airongl}@umich.edu 
Abstract 
Extractive summaries produced from multiple 
source documents suffer from an array of prob-
lems with respect to text cohesion.  In this pre-
liminary study, we seek to understand what 
problems occur in such summaries and how of-
ten.  We present an analysis of a small corpus 
of manually revised summaries and discuss the 
feasibility of making such repairs automati-
cally.  Additionally, we present a taxonomy of 
the problems that occur in the corpus, as well 
as the operators which, when applied to the 
summaries, can address these concerns.  This 
study represents a first step toward identifying 
and automating revision operators that could 
work with current summarization systems in 
order to repair cohesion problems in multi-
document summaries. 
1 Introduction 
With the increasing availability of online news 
sources, interest in automatic summarization has 
continued to grow in recent years.  Many systems 
have been developed for this purpose, including 
those that can produce summaries based on several 
documents (Multi-document summarization, or 
MDS).  Generally speaking, most of these systems 
work by extracting sentences from the original 
texts.  Although significant improvements continue 
to be made to such summarizers, they still cannot 
produce summaries that resemble those written 
manually by humans.  One area in particular in 
which the automatically produced summaries dif-
fer markedly is text cohesion. 
Whether a summary is produced from one or 
more documents, important context may be ex-
cluded from the summary that disrupts its readabil-
ity.  A text is not a random collection of sentences, 
but rather, each sentence plays a role in conveying 
the ideas the author wants to express.  Selecting 
sentences from multiple texts one at a time disre-
gards this interdependence between sentences.  As 
a result, summaries often suffer from problems of 
cohesion. 
1.1 Text cohesion and coherence 
[Halliday & Hasan, 1976] offer a clear definition 
for text cohesion: 
[The concept of cohesion] refers to relations of 
meaning that exist within the text, and that define it 
as a text.  Cohesion occurs where the interpretation 
of some element in the discourse is dependent on 
that of another (p.2). 
It is this property of cohesion that allows the 
reader to comprehend the overall meaning of a 
text, and to understand the author’s intentions.  
Therefore, in automatically produced summaries, 
cohesion problems should be resolved.  Otherwise, 
the resulting text may be unintelligible, or worse, 
misleading. 
1.2 Problems of cohesion in automatically pro-
duced summaries 
The following is an example of a summary pro-
duced automatically from one source document. 
 [1] More than 130 bodies are reported to have 
been recovered after a Gulf Air jet carrying 143 
people crashed into the Gulf off Bahrain on 
Wednesday.  [2] Distraught relatives also gathered 
at Cairo airport, demanding information.  [3] He 
also declared three days of national mourning. [4] 
He said the jet fell “sharply, like an arrow.” 
The most obvious problem with this summary is 
that in the last two sentences, the pronouns have no 
       Philadelphia, July 2002, pp. 27-36.  Association for Computational Linguistics.
          Proceedings of the Workshop on Automatic Summarization (including DUC 2002),
antecedents; as a result, the reader does not know 
who the subjects of the sentences are.  In addition, 
the adverb ‘also,’ used in both the second and third 
sentences, makes reference to previous events not 
described in the summary.  Another concern is that 
there seems to be no transition between the sen-
tences.  The context from the source article neces-
sary to make the text cohesive is missing from the 
summary.  As a result, the summary is unintelligi-
ble. 
1.3 Text cohesion in MDS 
Using multiple documents to generate a summary 
further complicates the situation.  As contended by 
[Goldstein et al, 2000] a multi-document summary 
may contain redundant messages, since a cluster of 
news articles tends to cover the same main point 
and shared background.  In addition, articles from 
various sources could contradict one another, as to 
how or when an event developed.  Finally, since 
the source articles are not all written simultane-
ously, they may describe different stages of the 
development of an event.  Not only do news stories 
come to different conclusions at various stages in 
an event, but also the attitudes of writers may 
change. 
Multi-document summaries may suffer further 
from problems of cohesion since their source arti-
cles may be written by different authors.  Not only 
do writers have their own styles, they have the 
overarching structure of the article in mind when 
producing it.  As a result, in MDS we are more 
likely to encounter text that is not cohesive. 
Previous research has addressed revision in 
single-document summaries [Jing & McKeown, 
2000] [Mani et al, 1999] and has suggested that 
revising summaries can make them more informa-
tive and correct errors.  We believe that a generate-
and-revise strategy might also be used in creating 
better multiple-document summaries, within the 
framework of current extractive summarization 
systems.  However, as mentioned previously, there 
is reason to believe that multi-document summa-
ries suffer from many different coherence prob-
lems and that such problems occur more often than 
in single-document summaries.  Therefore, an im-
portant preliminary step in determining how we 
might revise such summaries is to closely examine 
the cohesion problems that occur in multi-
document summaries. 
In the current paper we analyze a small corpus 
of manually revised multi-document summaries.  
We present a taxonomy of pragmatic concerns 
with respect to cohesion in the summaries, as well 
as the operators that can address them.  Finally, we 
will discuss the feasibility of implementing such 
revisions automatically, which we hope to address 
in our future work. 
2 Background and previous work 
2.1 Theories on discourse structure 
Rhetorical Structure Theory (RST) [Mann & 
Thompson, 1988] has contributed a great deal to 
the understanding of the discourse of written 
documents.  RST describes the coherence nature of 
a text and is based on the assumption that the ele-
mentary textual units are non-overlapping text 
spans.  The central concept of RST is the rhetorical 
relation, which indicates the relationship between 
two spans. 
RST can be used in sentence selection for sin-
gle document summarization [Marcu, 1997]. 
However, it cannot be applied to MDS.  In RST, 
text coherence is achieved because the writer in-
tentionally establishes relationships between the 
phrases in the text.  This is not the case in MDS, 
where sentences are extracted from different 
source articles, written by various authors. 
Inspired by RST, [Radev, 2000] endeavored to 
establish a Cross-document Structure Theory 
(CST) that is more appropriate for MDS.  CST fo-
cuses on the relationships between sentences that 
come from multiple documents, which vary sub-
stantially from those between sentences in the 
same text.  Such relationships include identity, 
paraphrase and subsumption (one sentence con-
tains more information than the other). 
2.2 Computational models of text coherence 
Based on RST, [Marcu, 2000] established a 
Rhetorical Parser.  The parser exploits cue phrases 
in an algorithm that discovers discourse 
relationships between phrases in a text.  This 
parser can be used to extract sentences in single-
document summarization.  To contrast, 
[Harabagiu, 1999] concentrated on the derivation 
of a model that can establish coherence relations in 
a text without relying on cue phrases.  She made 
use of large lexical databases, such as Wordnet, 
and of path finding algorithms that generate the 
algorithms that generate the cohesion structure of 
texts represented by a lexical path. 
[Hovy, 1993] summarized previous work that 
focused on the automated planning and generation 
of multi-sentence texts using discourse relation-
ships.  Text generation is relevant to MDS, as we 
can view MDS as an attempt to generate a new text 
by reusing sentences from different sources.  The 
systems discussed in [Hovy, 1993] relied on a 
knowledge base and a representation of discourse 
structure.  The dependency of text generation on 
knowledge of discourse structure was emphasized. 
2.3 Revision of single-document summaries 
[Mani et al, 1999] focused on the revision of sin-
gle-document summaries in order to improve their 
informativeness.  They noted that such revision 
might also fix ‘coherence errors.’  Three types of 
revision operators were identified: sentence com-
paction, sentence aggregation and sentence 
smoothing.  To contrast, [Jing & McKeown, 2000] 
concentrated on analyzing human-written summa-
ries in order to determine how professionals con-
struct summaries.  They found that most sentences 
could be traced back to specific cut-and-paste op-
erations applied to the source document.  They 
identified six operations and used them to imple-
ment an automatic revision module. 
2.4 Temporal ordering of events 
[Filatova & Hovy, 2001] addressed the issue of 
resolving temporal references in news stories.  Al-
though events in articles are not always presented 
in chronological order, readers must be able to re-
construct the timeline of events in order to com-
prehend the story.  They endeavored to develop a 
module that could automatically assign a time 
stamp to each clause in a document.  Using a syn-
tactic parser, patterns were discovered as to which 
syntactic phrases tend to indicate the occurrence of 
a new event.  In MDS, the correct temporal rela-
tionships between events described in the extracted 
sentences often needs to be reestablished, since 
they may be incorrect or unclear. 
[Barzilay et al, 2001] evaluated three algo-
rithms for sentence ordering in multi-document 
summaries.  One algorithm implemented was the 
Chronological Ordering algorithm.  However, the 
resulting summaries often suffered from abrupt 
changes in topic.  After conducting an experiment 
in which they studied how humans manually or-
dered sentences in a summary, they concluded that 
topically related sentences should be grouped to-
gether.  The Chronological Ordering algorithm was 
augmented by introducing a cohesion constraint.  
The evaluation of the output summaries demon-
strated a significant improvement in quality. 
3 Revision-based system architecture 
The proposed architecture of our system, which 
would implement the generate-and-revise approach 
to summarization, is depicted in Figure 1.  Input to 
this system is a cluster of source documents related 
to the same topic.  Next, sentence extraction takes 
place, in which important sentences in the articles 
are identified.  The output of this module is an ex-
tract, which lists the sentences to be included in the 
summary. 
In the next stage, Cross-document Structure 
Theory (CST) relationships are established.  Spe-
cific relationships between sentences are identified.  
Here, a CST-enhancement procedure [Zhang et al, 
2002] may take place, ensuring that interdependent 
sentences appear together in a summary.  Sen-
tences may also be reordered in the summary with 
respect to their temporal relations, topic, or other 
criteria. 
The next stage in the process is the revision 
module.  First, high level revision operators are 
chosen, with respect to the cohesion problems that 
need repair.  Afterwards, the specific lexical items 
to be added, deleted or modified are chosen.  The 
output of this module is the revised, enhanced 
summary. 
3.1 The MEAD summarizer  
The MEAD summarizer [Radev et al, 2000] 
[Radev et al 2002] is based on sentence extraction 
and uses a linear combination of three features to 
rank the sentences in the source documents.  The 
first of the three features is the centroid score, 
which quantifies the centrality of a sentence to the 
overall cluster of documents.  The second is the 
position score, which assigns higher scores to sen-
tences that are closer to the beginning of the docu-
ment.  The third feature, length, gives a higher 
score to longer sentences.  Using a linear combina-
tion of the three features, sentences are ranked by 
score and added to the summary until the desired 
length is attained. 
Summarization
CST Identification
C:1 - B:2
B:13
D:5 - A:1
Sentence Reordering
Temporal, Topical
Revision
High-level Operators
Lexical Choice
A:1 - 1. <Delete> time exp
2. Thursday
C:1
B:2
D:5 -1. <Add> adverb
2. Meanwhile
Multi-Document
Extract
CST Enhanced
Summary
CST Enhanced
Revised Summary
Source
Documents
A
B
C
D
C:1
B:13
D:5 
 
Figure 1: Revision-based MDS architecture: 
Letters denote documents; numbers denote sen-
tence numbers (within documents) 
4 Data and procedure 
We generated a corpus of summaries using the 
MEAD summarizer.  The original documents come 
from three sources – DUC 2001, the Hong Kong 
News corpus, and the GA3-11 data set.  One clus-
ter of related news articles was chosen from each 
source.  The DUC 2001 articles describe the 1991 
eruption of Mount Pinatubo in the Philippines.  
This cluster, which is not typical of the DUC data, 
focuses on this single event and its subevents over 
a 2-week time period.  Those taken from the HK 
corpus are about government initiatives surround-
ing the problem of drug rehabilitation.  Due to the 
expense and labor involved in the generation and 
revision of multi-document summaries, we have 
used a subset of 15 summaries from our corpus in 
order to develop our revision taxonomy and to pre-
sent some initial findings.  Our future revision 
studies will employ a much larger set of data. 
The summaries were revised manually by the 
first author.  This was a three-step process that in-
volved identifying each problem, choosing an 
operator that could address the problem and then 
selecting the lexical items to which the operator 
should be applied.  It is important to note that mul-
tiple lexical choices are possible in some cases. 
Since we were interested in identifying all 
types of cohesion problems as well as considering 
all possibilities for addressing these problems, the 
reviser was permitted to make any revision neces-
sary in order to correct problems in the summaries.  
Obviously, a module that makes revisions auto-
matically would be much more restricted in its set 
of revision operators.  However, since a major goal 
for this paper was to establish a taxonomy of prob-
lems specific to multi-document summarization 
and to consider the complexities involved in mak-
ing repairs in MDS, we did not place such restric-
tions on the reviser.  Rather, she applied 
corrections to the summaries as to make them as 
intelligible as possible, given the sentences chosen 
by the summarizer. 
 
Source Length  
(sentences) 
#Source 
documents 
DUC 2001 3 3 
DUC 2001 3 3 
DUC 2001 5 3 
DUC 2001 6 5 
DUC 2001 9 5 
GA3-11 3 3 
GA3-11 3 5 
GA3-11 6 5 
GA3-11 8 3 
GA3-11 7 3 
HK-125 3 3 
HK-125 5 3 
HK-125 6 5 
HK-125 5 5 
HK-125 8 3 
Table 1: Summaries from training data 
4.1 Revision example 
<DELETE-place stamp> Cairo, Egypt – </DELETE> The 
crash of a Gulf Air flight that killed 143 people in Bahrain 
<ADD-time exp-day>Wednesday </ADD> is a disturbing 
déjà vu for Egyptians: It is the second plane crash within a 
year to devastate this Arab country.  Egypt, which lacks the 
oil wealth of the Gulf and has an economy struggling to re-
vive from decades of socialist stagnation, has a long tradi-
tion of sending workers to the Gulf to fill everything from 
skilled to menial jobs.  <DELETE-place stamp> Manama, 
Bahrain (AP) – </DELETE> <ADD-time exp-day> On Fri-
day, </ADD> three bodies wrapped in cloth, one the size of 
a small child, were lain before the faithful in the Grand 
Mosque during a special prayer for the dead in honor of the 
<DELETE-redundancy> 143 </DELETE> victims of the 
<DELETE-overspecified entity> Gulf Air </DELETE> 
crash.  Bahrain’s Prime Minister Sheik Khalifa bin Salman 
Al Khalifa and other top officials stood side-by-side with 
2,000 Muslins reciting funeral prayers before the bodies, 
<DELETE-redundancy> which were among the 107 adults 
and 36 children killed in Wednesday’s air disaster, 
</DELETE> said Information Ministry spokesman Syed el-
Bably. 
Figure 2: Revised multi-document summary 
The above figure shows an example of a revised 
summary that was produced from three source arti-
cles from the GA3-11 corpus.  The news stories 
were collected live from the web, and come from 
two different sources www.foxnews.com and 
www.abcnews.com.  The revision operator used 
and the corresponding pragmatic concern precede 
the modified text in pointed brackets.  This type of 
markup scheme was used because it enables us to 
use simple Perl scripts to move between the origi-
nal and revised versions of the summaries. 
5 Taxonomy of revision strategies 
Based on our corpus of revised summaries, we 
have identified five major categories of pragmatic 
concerns related to text cohesion in multi-
document summaries: 
1) Discourse – Concerns the relationships be-
tween the sentences in a summary, as well as 
those between individual sentences and the 
overall summary. 
2) Identification of entities – Involves the reso-
lution of referential expressions such that 
each entity mentioned in a summary can eas-
ily be identified by the reader. 
3) Temporal – Concerns the establishment of 
the correct temporal relationships between 
events. 
4) Grammar – Concerns the correction of 
grammatical problems, which may be the re-
sult of juxtaposing sentences from different 
sources, or due to the previous revisions that 
were made. 
5) Location/setting – Involves establishing 
where each event in a summary takes place. 
Explanations of the specific pragmatic concerns in 
each category, as well as their corresponding op-
erator(s), are detailed in the appendix.  Overall, 
160 revisions were made across the 15 summaries. 
 
Pragmatic 
category 
# of revisions 
% of total 
revisions 
Discourse 54 34% 
Entities 41 26%
Temporal 35 22% 
Grammar 20 12%
Place/setting 10 6% 
Table 2: Revisions by pragmatic category 
The majority (82%) of the revisions fall into 
the first three categories.  This is not surprising, as 
in MDS, we expect to find many problems relating 
to discourse – such as abrupt topic shifts or redun-
dant messages.  Additionally, concerns relating to 
the identification of entities in the text are likely to 
occur when the sentence from the original docu-
ment that introduced an entity is not included in 
the resulting summary, but sentences that make 
reference to the entity are included.  Finally, it may 
not be clear when events described in a summary 
occurred.  This could be because sentences which 
stated when the event occurred were left out of the 
summary or because the sentences include relative 
time expressions such as ‘today’ even though the 
stories were written at different times or on differ-
ent days. 
Revisions relating to grammar or to establish-
ing where an event occurred were less frequently 
used, accounting for only 12% and 6% of the total 
repairs, respectively.  Sentences extracted from the 
original news stories are usually grammatical.  
However, problems related to grammar may arise 
from previous revisions.  In our corpus, the place 
or setting of an event was typically obvious in the 
summary and rarely required repair. 
Next, we present the analysis of revisions 
within each of the five categories.  We are inter-
ested in revising our summaries to be as coherent 
as possible, without having to implement compli-
cated and knowledge-intensive discourse models.  
Therefore, we will discuss the feasibility of im-
plementing the revisions in our taxonomy auto-
matically. 
5.1 Discourse-related concerns in MDS 
It is intuitive that problems relating to discourse 
are abundant in our summaries and, at the same 
time, that such repairs would be the most difficult 
to make.  The first obstacle is the detection of each 
of these concerns, which requires knowledge of the 
rhetorical relations of the sentences in the sum-
mary. 
 
Problem Number (%) 
1) Topic shift 24  (45%) 
2) Purpose 18  (33%) 
3) Contrast   6  (11%) 
4) Redundancy   6  (11%) 
5) Conditional   0 
Total 54 
Table 3: Discourse-related revisions 
In all the instances of topic shift and lack of pur-
pose in our corpus, a phrase or an entire sentence 
was added to provide a transition or motivation for 
the troublesome sentence.  Therefore, our module 
would require the ability to generate text, in order 
to repair these problems, which occur often in our 
summaries. 
5.2 Identification of entities in MDS 
Nine specific problems were found that concern 
the reader’s ability to identify each entity men-
tioned in a summary.  Most of these revisions 
could be made using rewrite rules.  For example, if 
it can be determined that a definite article is used 
when a (non-proper noun) entity is mentioned for 
the first time, the misused definite article could be 
replaced with the corresponding indefinite article. 
The most frequent problem, underspecified en-
tity, is the most difficult one to correct.  This dis-
fluency typically occurs where an entity is referred 
to by a proper noun or other noun phrase, such as 
the name of a person or organization, but has no 
title or further description.  In such cases, the miss-
ing information may be found in the source docu-
ment only.   
 
Problem Number (%) 
1) Underspecified entity 15  (38%) 
2) Misused quantifier   6  (15%) 
3) Overspecified entity   5  (12%) 
4) Repeated entity   5  (12%) 
5) Bare anaphor   4  (10%) 
6) Misused definite article   3  (  7%) 
7) Misused indef. Article   1  (  2%) 
8) Missing article   1  (  2%) 
9) Missing entity   1  (  2%) 
Total 41 
Table 4: Revisions concerning entity identification 
Therefore, to correct the underspecified entity 
problem, a revision module might require a knowl-
edge source for the profiles of entities mentioned 
in a summary.  When an entity is introduced for 
the first time in a summary, it should be associated 
with its description (such as a title and full name 
for a person). 
Discourse information would be useful for 
solving problems such as a bare anaphor or miss-
ing subject.  In revising single-document summa-
ries, [Mani et al, 1999] employed rules such as the 
referencing of pronouns with the most recently 
mentioned noun phrase.  However, this might be 
inappropriate in MDS, where the use of multiple 
documents increases the number of possible enti-
ties with which an anaphor could be referenced. 
5.3 Temporal relationships in MDS 
An important aspect of revision in MDS is the es-
tablishment of the correct temporal relationships 
between the events described in a summary.  We 
have identified five types of problems that fall into 
this category. 
 
Problem Number (%) 
1) Temporal ordering 31 (89%) 
2) Time of event 2    (6%) 
3) Event repetition 1   (2.5%) 
4) Synchrony 1   (2.5%) 
5) Anachronism 0 
Total 35 
Table 5: Temporal relationships revisions 
The most frequent revision in this category for 
our multi-document summaries was temporal or-
dering.  This is an important consideration for the 
summarization of news articles, which typically 
describe several events or a series of events in a 
given news story. 
A revision module might use metadata, includ-
ing the time stamps of source documents, in addi-
tion to surface properties of sentences in 
addressing this problem.  Temporal relations were 
typically established by adding a time expression 
to one or more sentences in a summary.  Therefore, 
our module will require a dictionary of such ex-
pressions as well as a set of rules for assigning an 
appropriate expression to a given sentence.  For 
example, if the time stamps of two source docu-
ments from which two adjacent summary sen-
tences come indicate that they were written one 
day apart, an appropriate way to order them might 
be: add a time expression indicating the day to the 
first sentence, and a relative time expression such 
as ‘the following day’ to the second sentence.  Our 
dictionary will require both relative and absolute 
time expressions at different levels of granularity 
(hour, day, etc.). 
Most of the temporal revisions in our corpus 
were made at points where sentences from differ-
ent sources followed one another or when sen-
tences from the same source were far apart in the 
original document.  By using such clues, it is 
hoped that temporal relations problems in summa-
ries can be corrected without knowledge of the 
discourse. 
5.4 Grammatical concerns in MDS 
The majority of grammatical problems in our cor-
pus resulted from previous revisions performed on 
the text.  For example, the addition of information 
to a sentence can result in it becoming too long.  
Such concerns can also occur because the grammar 
of one sentence, such as verb tense, does not match 
that of the next sentence. 
 
Problem Number  (%) 
1) Run-on sentence 7  (35%) 
2) Mismatched verb 3  (15%) 
3) Missing punctuation 3  (15%) 
4) Awkward syntax 3  (15%) 
5) Parenthetical 2  (10%) 
6) Subheading/titles 1  (  5%) 
7) Misused adverb 1  (  5%) 
Total 20 
Table 6: Grammatical revisions 
A revision module should be able to correct 
the above concerns using rules applied after other 
revisions are made and without any discourse 
knowledge. 
5.5 Location/setting concerns 
The least frequent type of revision made in our 
corpus related to establishing the correct locations 
of events in a summary.  Occasionally, a sentence 
in a summary retains the place/source stamp that 
appears at the beginning of a news article.  This 
appears ungrammatical unless the sentence is the 
first in the summary. 
Problem Number (%) 
1) Place/source stamp 6  (60%) 
2) Place of event 4  (40%) 
3) Collocation 0 
4) Change of location 0 
Total 10 
Table 7: Location/setting concerns 
In addition, such stamps might be inappropriate for 
a summary, since not all the sentences may share 
the same location.  In order to promote cohesion in 
the summary, our module could move the stamp 
information into the body of the summary. 
Sentences could be missing location informa-
tion altogether.  In such cases, the revision module 
might require information from the source docu-
ments in order to repair this problem.  Overall, the 
revisions related to establishing the location of 
events should not require knowledge of discourse 
in the summary.  Adding location information can 
usually be performed with the addition of a prepo-
sitional phrase, usually at the beginning of the sen-
tence. 
6 Conclusions and future work 
This paper represents preliminary work in our ef-
forts to address problems of text cohesion and co-
herence in multi-document summaries via revision.  
As a first step, we need to identify the specific 
problems that occur in MDS and consider how we 
might address such concerns.  To this end, we have 
investigated the optimal revisions that were per-
formed on a small set of summaries.  From this 
analysis, we have formulated a taxonomy of prag-
matic concerns and their operators for repairing 
multi-document summaries. 
Knowledge of 
discourse
and text generation 
<ADD>
transitional phrase or sentence;
<ADD>
motivational phrase or sentence;
Knowledge 
of discourse
Knowledge source
(entity 
descriptions)
Meta data
and dictionary 
of expressions
Sentence 
surface
structure
<DELETE>
Redundant information
<ADD/MODIFY>
discourse markers
<ADD/MODIFY> 
description of entity 
mentioned for first time
<ADD/MODIFY> 
Time expression
Grammar corrections;
<MODIFY/ADD/DELETE>
definite or indefinite articles;
<MODIFY/ADD/DELETE>
location of event
Op
er
a
t
ion   C
o
mplexit
y
 
Figure 3: Continuum of revision operations 
There is a scale of revision operations that can be 
performed (as shown in Figure 3), ranging from 
concrete repairs that require only knowledge of the 
surface structures of sentences, to knowledge-
intensive repairs that cannot be implemented with-
out a discourse model.  In the future, we plan to 
formalize our framework so that we might be able 
to implement such revision strategies automati-
cally.  Of course, such an automatic process will be 
much more constrained in the revisions it can ap-
ply, unlike the human reviser in our current study.  
For example, in automating the repair process we 
will be restricted to using only material from the 
source documents.  In addition, we may expand 
our taxonomy as necessary in exploring additional 
data.  We will need to relate revision in MDS to 
CST since revision required in a given summary 
depends on the relationships between sentences.  
Finally, we would like use the corpus of data we 
have collected to learn revision automatically. 
Acknowledgments 
The authors would like to thank Naomi Daniel, 
Hong Qi, Adam Winkel, Zhu Zhang and three 
anonymous reviewers for their helpful comments 
and feedback on this paper.  This work was par-
tially supported by the National Science Founda-
tion’s Information Technology Research program 
(ITR) under grant IIS-0082884. 
The version of MEAD that we used was de-
veloped at the Johns Hopkins summer workshop in 
2001 under the direction of Dragomir Radev.  We 
want to thank the following individuals for their 
work on MEAD: Sasha Blair-Goldensohn, John 
Blitzer, Arda Celebi, Elliott Drabek, Wai Lam, 
Danyu Liu, Hong Qi, Horacio Saggion and Simone 
Teufel.  

References 
[Barzilay et al, 2001]  Regina Barzilay, Noemie 
Elhadad, and Kathleen R. McKeown. Sentence 
ordering in multi-document summarization.  In 
Proceedings of HLT, San Diego, CA, 2001.  
[Filatova & Hovy, 2001]  Elena Filatova and Edu-
ard Hovy.  Assigning time-stamps to event-clauses.  
In Proceedings, ACL Workshop on Temporal and 
Spatial Information Processing, Toulouse, France, 
July 2001.  
[Goldstein et al, 2000]  Jade Goldstein, Mark Kan-
trowitz, Vibhu Mittal, and Jamie Carbonell.  Sum-
marizing text documents: sentence selection and 
evaluation metrics.  In Proceedings of the 22nd 
ACM SIGIR Conference on Research and Devel-
opment in Information Retrieval, Berkeley, CA, 
1999. 
[Halliday & Hasan, 1976]  M. Halliday and R. 
Hasan. Cohesion in English.  London: Longman, 
1976. 
[Harabagiu, 1999]  Sanda M. Harabagiu.  From 
lexical cohesion to textual coherence: a data driven 
perspective.  Journal of Pattern Recognition and 
Artificial Intelligence, 13(2): 247-265, 1999. 
[Hovy, 1993]  Eduard Hovy.  Automated discourse 
generation using discourse structure relations.  Ar-
tificial Intelligence 63, Special Issue on Natural 
Language Processing, 1993. 
[Jing & McKeown, 2000]  Hongyan Jing and 
Kathleen R. McKeown.  Cut and paste based text 
summarization. In Proceedings of the 1st Confer-
ence of the North American Chapter of the Asso-
ciation for Computational Linguistics 
(NAACL'00), Seattle, WA, May 2000. 
[Mani et al, 1999]  Inderjeet Mani, Barbara Gates, 
and Eric Bloedorn.  Improving summaries by re-
vising them.  In Proceedings of the 37th Annual 
Meeting of the ACL ’99, pages 558-565, Mary-
land, 1999. 
[Mann & Thompson, 1988]  William C. Mann and 
Sandra A. Thompson.  Rhetorical structure theory: 
toward a functional theory of text organization.  
Text, 8(3), 1988.   
[Marcu, 1997]  Daniel Marcu.  From discourse 
structure to text summaries.  In Proceedings of the 
ACL ’97 EACL ’97 Workshop on Intelligent Scal-
able Text Summarization, pages 82-88, Madrid, 
Spain, July 1997. 
[Marcu, 2000]  Daniel Marcu.  The theory and 
practice of discourse parsing and summarization, 
The MIT Press, November 2000. 
[Radev, 2000]  Dragomir Radev. A common the-
ory of information fusion from multiple text 
sources, step one: cross-document structure. In 
Proceedings, 1st ACL SIGDIAL Workshop on 
Discourse and Dialogue, Hong Kong, October 
2000. 
[Radev et al, 2000]  Dragomir R. Radev, Hongyan 
Jing and Malgorzata Budzikowska. Centroid-based 
summarization of multiple documents: sentence, 
extraction, utility-based evaluation, and user stud-
ies.  In ANLP/NAACL Workshop on Summariza-
tion, Seattle, WA, April 2000. 
[Radev et al, 2002]  Dragomir Radev, Simone Teu-
fel, Horacio Saggion, Wai Lam, John Blitzer, Arda 
Celebi, Hong Qi, Daniu Liu and Elliot Drabek.  
Evaluation challenges in large-scale multi-
document summarization: the MEAD project. 
Submitted to SIGIR 2002, Tampere, Finland, Au-
gust 2002. 
[Zhang et al., 2002]  Zhu Zhang, Sasha Blair-
Goldensohn, and Dragomir Radev. Towards CST-
enhanced summarization.  To appear in AAAI 
2002, August 2002. 
