AN INT~ATIONAL DELPHI POLL ON FUTURE TRENDS 
IN "INFORMATION LINGUISTICS" 
Rainer Kuhlen 
Universitast Konstanz 
Informationswissenschaft 
Box 6650 
D-7750 Konstanz I, West Germany 
ABSTRACT 
The results of an international Delphi poll on 
information linguistics which was carried out 
between 1982 and 1983 are presented. 
As part of conceptual work being done in 
information science at the University of Constance 
an international Delphi poll wss carried out from 
1982 to 1983 with the aim of establishing a 
mid-term pro@aosis for the development of 
"information linguistics". The term "information 
linguistics" refers to a scientific discipline 
combining the fields of linguistic data processing, 
applied computer science, linguistics, artificial 
intelligence, and information science. A Delphi 
poll is a written poll of experts - carried out in 
this case in two phases. The results of the first 
round were incorporated into the second round, so 
that participants in the poll could react to the 
trends as they took shape. 
I. Some demoscopic data 
I. I Return rate 
Based on sophisticated selection procedures 385 
international experts in the field of information 
linguistics were determined and were sent 
questionnaires in the first round (April 
1982). 90 questionnaires were returned. In the 
second round 360 questionnaires were mailed 
out (January 1983) and 56 were returned, 48 of 
these from experts who had answered in the first 
round. The last questionnaires were accepted at the 
end of June 1 983. 
Overlapping data in the two rounds 
first round (90) second round (56) 
2 48 8 
In the following we refer to four sets of data: 
Set A 90 from round I 
Set--B 48 from round I with answers in round 2 
8et--C 56 from round 2 
Set--D 48 from round 2 with answers in round I 
But we shall concentrate primarily on Set C becanse 
- according to the Delphi philosophy - t~e data of 
the second round are the most relevant. There 
were 8 persons within Set C who did not answer in 
the first round. But the~ also were aware of 
the results of the first round; therefore a 
Delphi effect was possible. (In the following the 
whole integers refer to absolute numbers; the 
decimal figures to relative/procentual numbers) 
I .2 Qualification accordin~ to academic degree 
The survey singled out highly competent people, as 
reflected in academic degree( data from A and C): 
Tab.1 Qualification of l~articipants 
Set A Set C 
B.S./B.A 23 25.6 16 28.6 
M.S./M.A./Dipl. 40 44.4 28 50.0 
Ph.D./Dr. 62 68.9 37 66.1 
Professor 14 15.6 15 26.8 
1.3 A~e 
Since Delphi polls are concerned with future 
developments, it has been claimed in the past that 
the age and experience of people in the field 
influence the rating. In this paper, however, we 
cannot prove this hypothesis. Here are the mere 
statistical facts, only taken from Set C (they do 
not differ significantly in the other--sets) 
Tab.2 Age of participants 
-30 30-35 36-40 41-45 46-50 50- years 
3 5.6 14 25.9 14 25.9 10 18.5 5 9.3 8 14.8 
I .4 Experience 
The number of years these trained specialists have 
been working in the general area of information 
linguistics were as follows 
Tab.3 Experience in information lin~istics 
-2 3-5 6-I0 I O- years of experience 
3 5.6 7 13.O 13 24.1 31 57.4 
$ 540 
These data in particular confirm our impression 
that very qualified and experienced people 
answered the questionnaire. Almost 60% have 
worked longer than 10 years in the general area 
of information linguistics. 
1.5 Size of research groups 
Mos~ of those answering the questionnaire work in a 
research-group. Table 4 gives an impression of the 
size of ~ne groups in SetA and Set_C: 
Tab.4 Size of research groups 
I-2 3-5 6-10 11-50 50 - 
Set A 16 19.0 25 29.8 21 25.0 18 21.4 4 4.8 
Se~--C 14 26.4 17 32.1 12 22.6 8 15.1 2 3.8 
1.6. Represented subject fields 
Among those answering in the two 
lowing fields were represented: 
rounds, the fol- 
Tab.5 Scientific back6round of participants 
Set A Set C 
information science 32 35.6 17 30.4 
computer science 36 40.0 20 35.7 
linguistics 21 27.3 16 28.6 
natural sciences/ 15 16.7 12 21.4 
mathematics 
e,ngineerin 6 3 3.3 2 3.6 
humanities/social 15 16.7 12 21.4 
sciences 
I~7 Research and application/development 
With respect to whether participants are mainly 
involved in research (defined as: basic 
groundwork, mainly of theoretical interest, 
experimental environment) or in applica- 
tion/development (defined as: mainly of interest 
from the point of view of working systems (i.e. 
commercial, industrial), applicable to routine 
tasks) the results were as follows: 
Tab.6 Involved in research or application 
Set A Set B Set C Set D 
research 59 65.6 31 64.6 39 69.6 33 68.8 
application 27 30.0 16 33.3 16 28.6 15 31.3 
1.8 Workin 6 environment 
Tab.7 Types of institutions 
Set A Set C 
m 
university 45 50.0 30 53.6 
research institute 7 7.8 4 7. I 
industrial research 17 18.9 12 21.4 
information industry 8 8.9 2 3.6 
indust, administ. - I I .8 
puolic administration 8 8.9 4 7.1 
public inf. systems 3 3.3 2 3.6 
Most of the work in information linguistics so far 
has concentrated on English ~generally more than 
80%, with slight differences in the single 
sub-areas, i.e. acoustic 80.6%, indexing 
82.5%, question-answering83.3%). 
2. Content of the ~uestionnaire 
2. I Sub-areas 
The discipline "information linguistics" was not 
defined theoretically but ostensively instead by a 
number of sub-areas. 
abreviation 
I. Acoustic/phonetic procedures Ac 
2. Morphological/syntactic procedures Mo 
3. Semantic/pr~m~tic procedures Se 
4. Contribution of new hardware Ha 
5. Contribution of new software So 
6. Information/documentation languages I1 
7. Automatic indexing In 
8. Automatic abstracting Ab 
9. Automatic translation Tr 
10. Reference and data retrieval systems Re 
11. Question answering and understanding Qu 
systems 
2.2 Single topics 
The sub-areas included a varying number of topics 
(from 6 to 15). These topics were chosen based 
on the author's experience in information linguis- 
tics, on a pretest with mostly German researchers 
and practitioners, on advices from members of 
FID/LD, and on long discussions with Don Walker, 
Hans Karlgren, and Udo Hahn. Altogether, there 
were 91 topics in the first round and 90 in the 
second round, as follows:. 
acl Segmentation of Acoustic Input 
ac2 Speaker Dependent Speech Recognition 
ac3 Speaker Independent Speech Recognition 
ac4 Speech Understanding 
ac5 Identification of Intonational/Prosodic Infor- 
mation with respect to Syntax 
ac6 Identification of Intonational/Prosodic Infor- 
mation with respect to Semantics 
ac7 Automatic Speech Synthesis 
mol 
mo2 
mo3 
mo4 
mo5 
mo6 
mot 
mo8 
mo9 
mol 0 
mol I 
Automatic Correction of Incomplete or False 
Input 
Analysis of Incomplete or Irregular Input 
Morphological Analysis (Reduction Algorithms) 
Automatic Determination of Parts of Speech 
Automatic Analysis of ?unctions& Notions 
Partial Parsing Recognition Techniques 
Partial Parsing Transformation Techniques 
Recognition of Syntactic Paraphrases 
Reco~ition of Textual Paraphrases 
Question Recognition 
Grits of Syntactic Parsing of Unrestricted 
Natural Language Input 
sel Semantic Classification of Verbs or Predicates 
se2 Or6mnizin6 Domain-Specific ?tame/Script-Type 
Structures 
se3 Semantically Guided Parsing 
se4 Semantic Parsing 
541 
se5 Knowledge Acquisition 
se6 Analysis of Quantifiers 
se7 Analysis of Deictic Expressions 
se8 Analysis of Anaphoric/Cataphoric Expressions 
(Pronominalization) 
se9 Processing of Temporal Expressions 
se10 Establishment of Text Cohesion and Text 
Coherence 
sel I Recognition of Argumentation Patterns 
se12 Management of Vague and Incomplete Knowledge 
set3 Automatic Management of Plans 
set4 Formalizing Speech Act Theory 
se15 Processing of "Unpr~m~tical" Input 
hal Personal Computers for Linguistic Procedures 
ha2 Parallel Processing Systems 
ha3 New Mass Memory Technologies 
ha4 Associative Memory 
ha5 Terminal Support 
ha6 Hardware Realization of NatnAral Langusge 
Analysis Procedures 
ha7 Communication Networks 
sol Standard Progr~,mi ng Languages for Information 
Linguistics 
so2 Development of Modular Standard Programs 
(Hardware-Independent) 
so3 Natural Language ProgrPJ,ming 
so4 Parallel Processing Techniques 
so5 Alternative File Organization 
so6 New Database System Architecture for the 
Purpose of Information Linguistics 
so7 Flexible Data Management Systems 
i11 Compatibility of Documentation Languages in 
Distributed Networks 
il2 Enrichment of Information Languages by 
Statistical Relations 
ll3 Enrichment of Information/Documentation 
La~s by Linguistic Semantics 
il4 Enrichment of Higher Documentation Languages 
by Artificial Intelligence Methods 
il5 Standardization of Information/Documentation 
Languages 
il6 Documentation Languages for Non-Textual Data 
il7 Information/Documentation Languages for 
Heterogeneous Domains 
lib Determination of Linguistic Relations 
il9 Adaptation of Ordinary Language Dictionary 
Databases 
ill0 (cancelled in the second round) 
ill I Statistical Models of Domain-Specific 
Scientific Languages 
inl Improvement of Automatic Indexing by 
Morphological Reduction Algorithms 
in2 Improvement of Automatic Indexing by 
Syntactic Analysis 
in3 Improvement of Automatic Indexing by 
Semantic Approaches 
in4 Probabilistic Methods of Indexing 
in5 Indexing Functions 
in6 Automatic Indexing of Full-texts 
abl Abstracting Methodolo~ 
ab2 Automatic Extracting 
ab3 Automatic Indicative Abstracting 
ab4 Automatic Informative Abstracting 
ab5 Automatic Positional Abstracting ab6 Graphic Representation of Text Structures 
trl Development of Sophisticated Multi-Lingual 
Lexicons 
tr2 Automatic Translation of Restricted Input 
tr3 Interactive Translation Systems 
tr4 Fully Automatic Translation Systems 
tr5 Multilingual Translation Systems 
tr6 Integration of Information and Translation 
Systems 
rel Iterative Index and/or Query Modification 
by Enrichment of Term Relations 
re2 Natural Language Front-End to Database Systems 
re3 Graphic Displsy for Query Formulation support 
re4 Multi-Lingual Databases and Search Assistance 
re5 Public Information Systems 
qul Integration of Reference Retrieval and 
Question Answering Systems 
qu2 Linguistic Modeling of Question/Answer 
Interaction 
qu3 Formal Dialogue Behavior 
qu4 Belief Structures 
qu5 Heuristic/Common Sense Knowledge 
qu6 Change of Roles in Man-Machine Communication 
qu7 Automatic Analysis of Phatic Expressions 
qu8 Inferencing 
qu9 Variable Depth of System Answers 
qu10 Natural Language Answer Generation 
Each topic was defined by textual paraphrase, 
e.g. for ab4: "procedures of text condensation 
that stress the overall, true-to-scale compression 
of a given text; although varyin~ in length 
(according to the degree of reduction); can be used 
as a substitute for original texts". 
3. Answer parameters for the sub-areas 
3.1 Competence (--CO) 
At the beginning of every sub-area participants 
were requested to rate their competence accord- 
ing to three parameters "good" (with a 
speciaiist's knowledge), "fair" (with a 
working knowledge), and "superficial" (with a 
layman's knowledge). Tab.8 shows the 
self-estimation of competence within the sub-areas 
(data taken from SetC): 
Tab. 8 Competence Tab.9 Desirability 
good fair superficial ++ + 
rank rank rank 
Ac 4 11 14 8 34 1 
Mo 25 3 17 5 8 7 
Se 24 4 17 5 10 5 
Ha 13 10 23 \] 14 3 
So 18 7 22 2 8 7 
I1 18 7 18 4 12 4 
In 21 6 17 5 9 6 
Ab 14 9 20 3 16 2 
Tr 24 4 5 11 O 11 
Re 31 2 12 10 8 7 
Qu 32 1 13 9 7 10 
In 19 19 1 0 
Ab 21 22 4 O 
Tr 33 11 I 0 
Re 35 13 O 0 
Qu 35 83 0 
542 
3.2 Desirability (=DE) 
With respect to the application oriented subject 
areas the category of desirability was used in 
order to determine the social desirability 
according to the following 4-point scale: "very 
desirable"/++ (will have a positive social effect, 
little or no negative social effect, extremely 
beneficial), "desirable"/+ (in general positive, 
minor negative social effects), "undesirable"/- 
(negative social effect, socially harmful), "very 
ur~esirable"/m (major negative social effect, 
socially not justifiable). 
Tab.9 (data from Set C) shows that the nega- 
tive parameters (--, -)--were never or only sel~om 
used. Information linguistics is not judged - 
accordir~ to the estimation of the experts - as a 
socially harmful scientific discipline. 
4. Answer parameters for the single topics 
The following parameters were used as ratin~ for 
the sub-areas and the single topics. Their 
definitions were given in more detail in the 
questionnaire. 
Tab.10 Evaluation l~arameters 
IMPORTANCE(=I) FEASIBILITY(=F) DATE OF REALIZ. (=DR) 
~+ very i. ++ def. f. realized 
+ i. + poss. f. 1984+/-2 
1989 +/-3 
1996 +/-10 
- slightly i. - doubtf, f. 2010 +/-10 
w-un-i. --def. un-f. non-realistic 
These categories of scientific importance, 
feasibility, and date of realization were to be 
judged from tu~o points of view: 
research(=R) - defined as: basic groundwork, mainly 
of ~heoretical interest 
application/development (=A) - defined as: mainly 
of interest for working systems, applicable to 
routine tasks 
Therefore every single topic was evaluated accord- 
ing to six parameters: 
Importance for research I/R 
Importance for application I/A 
Feasibility for research F/R 
Feasibility for application A/A 
Date of realization considering research DR/R 
Date of realization considering application DR/A 
5. More detailed results 
5 • I Sub-areas 
5.1.1 Competence 
Competence was an important influence on evalua- 
tion. In general one can say that people 
with "good" competence (or more correctly: with 
competence estimation of "good") in a sub-area gave topics higher ratings for importance and 
feasibility both from the research and the 
application points of view. Nevertheless, there 
were differences. Those with "good" competence 
differed more widely in evaluations of 
research-oriented topics than in applica- 
tion-oriented topics, whereas those with "super- 
ficial" competence in the sub-areas were closer to 
the average in their evaluations of applica- 
tion-oriented topics than of research-oriented 
topics. Here are some examples of the differences 
(as reflected in the averages of the sub-areas). 
Tab. 11 is to be read as follows: (line I) in the 
sub-area "Acoustic" those with "good" competence 
evaluated 5.6% higher than the average with respect 
to importance for research, whereas people with 
"superficial" competence in the same sub-area 
evaluated 6.9% lower than average. 
Tab.11 Competence differences 
( g=good; s=superficial) 
I/R I/A F/R F/A CO/g CO/s CO/g CO/s CO/g CO/s CO/g CO/s 
Ac5.6+ 3.0- In4.7+ 5.1- Ac25.1+ 3.9- Ac9.4+ 0.6- 
Hal .8+ 9.3- Ab4.3+ 13.8- Sel .I- 5.8+ Ha7.5+ 7.0- 
In5.4+ 19.8- In6.2+ 19.4- In5.0+ 19.4- 
Ab7.2+ 8.4- 
As can be seen in the column F/R, sometimes the 
general trend is reversed (Semantic: values from 
"competent" participants are lower than from par- 
ticipants with "superficial" competence). 
5.1.2 Desirability 
There is also a connection between desirability and 
the values of importance and feasibility. Those who 
gave high ratin~s for desirability (DE++) in 
general gave higher values to the single topics in 
the respective sub-areas, both in comparison to 
the average values and to the values of those who 
gave only high desirability (DE+) to a given 
sub-area. The differences between DE++ and DE+ are 
even higher than those between C/g und C/s. 0nly 
the F/R data in the translation and retrieval areas 
are lower for D++ than for D+, in all other cases 
the D++ values are higher. Some examples: 
Tab. 12 Desirability differences 
I/R I/A F/R F/A 
DE++ DE+ DE++ DE+ DE++ DE+ DF~-* DE+ 
In 6.6+ 4.3- 4.5+ 4.9- 6.9+ 10.9- 11.4+ 15.3- 
Ab 6.8+ 0.6-13.2+ 5.8- 0.9+ 0.2+ 7.9+ 4.3- 
Tr 2.8+ 5.9- 0.4+ 1.1- 2.1- 8.3+ 2.9+ 3.2- 
Re 1.9+ 8.3- O.1+ - 0.2- 0.6+ 2.0+ 4.1- 
Qu4.O+ 8.1- 7.5+ 14.2- 3.8+ 11.4- 7.7+23.5- 
5.1.3 Importance, Feasibility, Date of Realization 
(In the following tables the values of the answers 
++ (very important, definitel~ feasible) and + 
(important, possibly feasible) have been added 
543 
~ ogether, and the values from the single topics ave oeen averaged. Exact year-datawere calcu- 
lated from the answers on the 6-point rating scale, 
cf. Tab.10. In order to show the Delphi effect 
the data in Tab. 13 are taken fromSet__A, in Tab.14 
from Set_C) 
Tab.13 Averaged I- r F- t DR-values from Set A 
Importance Feasibility Realization 
I/R I/A F/R F/A DR/R DR/A 
Ac 85.4 82.5 62.5 49.4 1997 2000 
Mo 84.0 87.7 84.1 75.9 1987 1990 
Se 89.2 81.2 67.5 53.3 1995 1999 
Ha 84.8 87.9 84.6 76.0 1986 1991 
So 88.1 88.9 80.8 72.1 1988 1994 
IL 77.6 79.0 83.1 74.6 1987 1993 
In 90.2 90.0 79.9 74.7 1986 1990 
Ab 79.8 77.7 69.2 58.7 1991 1997 
Tr 87.5 87. I 72.3 63.0 1994 1998 
Re 87.7 90.7 86.8 78.3 1 985 1 989 
Qu 87.5 80.2 74.2 61 .I 1991 19989 
Tab.14 Averaged I-, F- t DR-values from Set C 
I/R I/A F/R F/A DR/R DR/A 
Ac 90.9 84.0 64.2 46.4 1998 2001 
Mo 90.1 89.3 88.4 78.6 1967 1991 
Se 92.6 83.4 70.3 49.4 1996 2000 
Ha 82.4 83.8 88.6 75.8 1 987 1993 
So 88.0 88.3 80.1 67.5 1989 1996 
IL 82.8 83.4 88.0 77.0 1988 1997 
In 89.4 90.5 89.6 79.2 1986 1991 
Ab 75.6 75.0 68.8 52.3 1992 1999 
Tr 89.3 91.5 69.7 53.2 1994 2000 
Re 83.8 91.7 91.7 83.9 1986 1991 
Qu 88.4 80.8 76.8 52.7 1992 1999 
The average values in Tab. 13 and 14 should not 
be over-interpreted. In particular, ranking is 
unjustified. One cannot simply conclude that, 
say, the sub-area "Semantics" (92.6) is more 
important than that of "Abstracting" (75.6) with 
respect to research because the average value 
is higher; or that Indexing (79.2) is more 
feasible from an application point of view 
than Abstracting (52.3). $uch conclusions may be 
true, and this is why the values in Tab. 13 and 
14 are given, but the parameters should actually 
only be applied to the single topics in the 
sub-areas. Cross-group ranking is not allowed 
for methodological reasons. 
But nevertheless the 
It is obvious that 
general true: 
data are interesting enough. 
the following relation is in 
I/R (-values) > I/A > F/R > F/A 
There are some exceptions to this general rule, 
such as Re-I/A>I/R (both in Set A and Set C); 
Ha-F/R>I/R (in Set C); (Re-F/R ant F/A)>I/R--(in 
Set_C); and I1-F/R>~/R(both in Set_A and SetC). 
There seems to be a non-trivial g~p between impor- 
tance and feasibility (both with respect to 
research and application). In other words, there are more problems than solutions. And there is an 
even broader gap between application and research. 
From a practical point of view there is some skep- 
sis concerning the possibility of solving important 
research problems. And what seems to be feasible 
from a research point of view looks different from 
an application one. 
The values in the second round are in general 
higher than in the first one. This is an argument 
against the oft cited Delphi hypothesis that the 
feedback-mechanism - i.e. that the data of the 
previous round are made known at the start of the 
following round -has an averaging effect. The 
increase-effect can probably be explained by the 
fact that the percentage of qualified and "com- p 
etent" people was higher in the second round 
perhaps these were the ones who were motivated to 
take on the burden of a second round) - and, as 
Tab.11 shows, people who rated themselves "com- 
petent" tend to evaluate higher. 
Between the two rounds the decline in the 
sub-areas "Software" and "Hardware" (apart from the 
parameter F/R) is striking. There is an overall 
increase for '%lorphology" and "Information Lan- 
guages" for all parameters, and a dramatic increase 
for the topics in "Indexing" for F/R (9.7%), and a 
dramatic decline for the "Translation"- and "Ques- 
tion-Answering"-topics for the parameter F/A (9.8 
and 8.4%). 
The dates of realization do not change dramati- 
cally° On the average there is a difference of one 
year (and this makes sense because there was almost 
one year between round I and 2). There is a ten- 
dency from a research point of view for the expec- 
tation of realization to be somewhat earlier from 
an application standpoint. But the differences are 
not so dramatic as to justify the conclusion that 
researchers are more optimistic than 
developers/practitioners. 
5.2 Single topics 
Tab. 15 and 16 show the two highest rated topics in 
each sub-area in the first two columns and the two 
lowest rated topics in each sub-area in the last 
two columns. These represent average data from 
Set C. The four columns in the middle show the 
estimation of participants who work in research or 
application, respectively. As part of the demos- 
copic data it was determined whether participants 
work more in research or in application (cf. 
Tab.6). Notice that both groups answered from a 
research and application point of view. In a more 
detailed analysis (which will be published later) 
this- and other aspects- can be pursued. In 
Tab.15 and 16 the data for very high importance 
(*+) and high importance (+) have been added 
together. 
544 
Tab.15 Topics accordin 6 to importance 
most important topics (++^+) less important 
average research application average(--~-) 
I/R I/A I/R I/A I/R I/A I/R I/A 
acl ac7 acl acl acl a22 
ac3 ac2 ac3 ac2 ac2 ac3 
mc8 mol too8 tool too8 mol 
mo11 mo10 mo11 too3 mo9 mo2 
se5 se3 se5 se3 se2 se2 
se2 set2 se8 se2 se3 se5 
ha7 ha7 ha4 ha3 ha7 ha5 
ha4 ha5 ha2 ha7 ha2 ha7 
so6 so7 so6 so5 so3 so4 
so7 so5 so5 so7 so4 so6 
i110 i110 i14 i11 i11 i11 
i14 i11 i11 i14 i17 i16 
in3 inl in3 in6 in3 in3 
in2 in6 in6 in3 in6 in6 
ab4 ab3 ab4 ab2 ab3 ab3 
ab5 ab2 ab5 ab3 abl ab4 
tr3 tr3 tr2 tr3 tr3 trl 
tr5 tr2 tr5 tr2 tr4 tr3 
re2 rel re2 tel tel tel 
rel re5 rel re2 re2 re5 
qu5 qul qu2 qul qul qul 
qu2 qu8 qu5 qu8 qu5 qu2 
ac6 ac6 
ac7 ac5 
tool mo9 
too7 too4 
se15 set5 
se7 sel I 
ha6 ha6 
hal ha2 
sol so3 
so3 so4 
il5 ill I 
ill I il5 
in4 in5 
in5 in4 
ab2 ab6 
ab6 ab5 
trl tr5 
tr6 trl 
re3 re3 
re4 re4 
qu7 qu7 
qu3 qu3 
Tab. 16 Most feasible~ less feasible topics 
most feasible topics (++^+) less feasible 
average research application aversge(--A-) 
F/R F/A F/R F/A F/R WA F/R F/A 
ac7 ac7 ac2 ac7 ac2 ac2 
ac2 ac2 ac5 acl ac7 ac7 
too3 mo3 mo3 mo3 tool tool 
mot0 mot0 mot0 mot0 too2 mo2 
se3 se2 se3 se9 se2 se2 
se6 se6 se2 se2 se6 se6 
ha5 ha5 ha5 ha5 ha% ha4 
ha7 hal ha7 ha3 ha5 ha5 
so2 so2 so2 sol so2 so2 
sol sol sol so2 so7 so5 
i110 ill0 il9 il6 ill ill 
il9 il9 lib il9 il7 il7 
inl in4 in4 irg in3 in4 
in2 inl in5 in5 in4 in3 
ab2 ab2 ab2 ab2 ab2 ab2 
ab3 ab3 ab3 ab3 abl ab3 
tr3 tr3 tr3 tr3 tr3 tr3 
tr2 trl tr2 trl tr2 tr2 
rel re3 rel re3 rel rel 
re3 re5 re3 re5 re2 re3 
qul qul qul qul qul qu10 
qu2 qu10 qu2 qu10 qu5 qul 
ac6 ac6 
ac4 ac4 
mo9 mol I 
mo5 mo5 
set5 set5 
sell sel I 
has ha6 
ha2 ha2 
so3 so3 
so4 so4 
il7 il4 
i16 i15 
in6 in3 
in3 in6 
ab4 ab5 
ab5 ab6 
tr4 tr4 
tr5 tr5 
re4 re4 
re5 re2 - 
qu4 qu4 
qu9 qu9 
A final Table shows the data for short term and 
long term topics, only the two closest and the two 
most distant topics in each sub-area are given 
(data from SetC). 
Tab.l 7 1 ~o ~ term and lon 6 term t9pics 
short term long term 
R/R R/A R/R R/A 
ac7 1987 ac7 1992 
ac2 1991 ac2 1997 
too3 1984 mo3 1984 
mot0 1984 too6 1986 
se2 1987 sel 1992 
sel 1988 se6 1995 
ha5 1984 ha5 1985 
ha7 1984 ha3 1988 
sol 1984 sol 1987 
so2 1987 so2 1992 
il2 1986 il9 1990 
i±9 1986 il2 1991 
inl 1984 inl 1986 
in4 1984 in4 1987 
1986 ~ 1991 
~3 1988 ~3 1996 
at3 1985 at3 1990 
at2 1985 at2 1992 
re2 1984 re3 1987 
tel 1984 tel 1988 
qul 1988 qul 1997 
qu2 1988 qu2 1997 
ac4 2003 ac4 2006 
ac6 2003 ac6 2006 
too9 1997 too9 2000 
mo11 1992 mo11 1997 
set5 2000 sell 2005 
sel I 2000 se14 2005 
ha6 1996 ha6 1999 
ha2 1991 ha2 1997 
so3 1998 so3 2001 
so4 1993 so4 1998 
ill0 1989 il4 1997 
i15 1989 i13 1996 
in3 1 989 in3 1997 
in6 1988 in6 1997 
aa5 1996 sa4 2002 
aa6 1996 am6 2001 
at4 2000 at4 2006 
at5 1998 at5 2005 
re% 1992 re4 1998 
re5 1986 re5 1990 
qu9 1997 qu4 2001 
qu4 1997 qu5 2001 
Finally I would like to thank all those who par- 
ticipated in the Delphi rounds. It was an extremely 
time-consuming task to answer the questionnaire, 
which was more like a book than a folder. I hope 
the results justify the efforts. The analysis would 
not have been possible without the help of m~ 
colleagues - Udo Hahn for the conceptual desi~a, 
and Dr.J.Staud together with Annette Woehrle, Frank 
Dittmar and Gerhard Schneider for the statistical 
analysis. This project has been partially financed 
by the FID/LD-comnittee and by the "Bundesminis- 
terium fuer Forschung und Technologie/ Gesellschaft 
fuer Information und Dokumentation", Grant PT 
200.08. 
545 
