Powerful ideas in computational linquistics 
Implications for problem ,solvinq, and education 
Abstract 
Gerhard Fischer, Institut fuer Informatik 
Universitaet Stuttgart, West-Germany 
It is our firm belief that solving problems in 
the domain of computational linguistics (CL) 
can provide a set of metaphors or powerful 
ideas which are of great importance to many 
fields. We have taught several experimental 
classes to students from high schools and 
universities and s major part of our work was 
centered around problems dealing with language. 
We have set up an experimental Language 
Laboratory in which the students can explore 
existing computer programs, modify them, design 
new ones and implement them. The goal was that 
the student should gain a deeper understanding 
of language itself and that he/she should learn 
general and transferable problem solving 
skills. 
exercise in pattern matching and symbol 
manipulation, where certain keywords trigger a 
few prestored answers. It may also serve as an 
example for how little machinery is necessary 
to create the illusion of understanding. 
\[n our interdisciplinary research project 
(KLING eL el, 1977) we have tried to overcome 
these problems by providing opportunities for 
the student to explore powerful ideas in the 
context of non-trivial problems and by showing 
that the computer prescence can do much more 
for education than improve the delivery system 
for curricula established independently of it. 
I. Introduction 
Problem solving with the computer for the 
non-computer expert is slowly recognized as an 
important activity in our educational system. 
It is done best in a project-oriented course in 
which the student learns to solve problems in 
different domains. In the past, activities of 
this sort have been centered around numerical 
problems, physics problems and the standard 
computer science problems (eg like writing a 
sorting procedure). 
2. The state of the art 
The relevance of problems from linguistics has 
been ignored. The reasons for this fact are 
easy to explain: 
I) the educational community in the 
language-oriented fields has very little 
knowledge about using a computer to write 
interesting programs to gain a deeper 
understanding of the problems in their domain 
2) the computer experts were not familiar 
with linguistics 
3) the most commonly used programming 
lsnguegee end eyetemo ere inadequate to deal 
with the data structures and dialog 
requirements which are relevant for language 
processing 
4) new fields like artificial intelligence, 
cognitive science end computational 
llnguletlee were not widely known 
The level of ignorance can best be seen by 
using ELIZA as an example: many people thought 
that it was a program which would "understand" 
the contents of a dialog. It was not evident to 
them that ELIZA represents nothing more than an 
~. Coqnitive Science and Proqrammin~ 
In recent years the view has emerged that the 
language of computation is the proper dialect 
to describe basic issues in psychology, 
linguistics and education. Research in 
Cognitive Science has demonstrated that the 
phenomena surrounding computers are deep and 
obscure, requiring much experimentation. 
Cognitive Science theories about problem 
solving, representation of knowledge and other 
cognitive abilitiesprovide the foundation for 
our understanding of programming. 
We believe that the whole enterprise of 
programming can be much better explained with 
concepts from CL than with those from 
mathematics. Problems in CL are often 
ill-defined, algorithms are seldom given and 
programming is more a design task than it is s 
coding of a known algorithm. The problem 
formulation phase is more relevant than the 
execution of a Program and systems are needed 
to support this phase of the problem solving 
process. Successive formulation of programs 
serve aa stepping stones towards the goal of 
defining the specification of a problem. 
Humane have a good intuitive understanding of 
the problems in CL and they can do the things 
(like communicate in natural language, deal 
with vast amounts of knowledge, infer new 
knowledge from exiting one) - even if they do 
not know how they do it. Programming can be 
understood as an effort to make our own 
knowledge explicit and can provide us with 
adequate metaphors to describe our own mental 
functions. 
111 
4. Oesiqn of a Lanquaqe Laboratory 
The design of learning environments is an 
important goal for the educational theorist and 
the teacher. The computer as a new technology 
has created almost unlimited possibilities to 
create new and challenging environments. The 
Turtle world (PAPERT 1979) and the simulation 
world of Smslltslk (KAY 1977) provide good 
models of what can be done. 
In our project we have set up an experimental 
Language Laboratory in which the students can 
explore existing programs, modify them, dealgn 
new ones and implement them. We took great care 
in our design (by following the tradition of 
the LOGO projets as opposed to CAI approaches) 
that the students could work in an active mode 
end develop ideas in 8 personal way (not 
limited by the teachers approach). Our teaching 
style was not to provide answers but the 
learners were encouraged to use their own 
language knowledge to find e solution. Their 
work had to rely on self motivation which seems 
a more reasonable goal in CL where the products 
(eg poems, horoscopes, question/answering 
systems etc) can be more interesting and 
aesthetically pleasing then a set of numbers 
appearing as s result in numerical mathematics. 
With our Language Laboratory we wanted to 
create an environment in which the student's 
task is not to learn a set of formal rules (eg 
about the syntax of a programming language), 
but to give them s world in which they could 
develop sufficient inside into the way they 
used language to allow the transposition of 
this self-knowledge into programs. 
The students were exposed to different 
formalisms (primarily to LOGO, but also to 
LISP, ATNs, semantic networks, MICRO-PLANNER) 
and could explore the range of possible models 
which could be implemented in a cognitively 
efficient way with these formalisms. We tried 
to engage them in problems of moderate 
complexity (the students ware no researchern 
working full-time in a project) and we crested 
micro-verslons of programs by ommltlng feoturea 
which were not essential for a conceptual 
understanding. 
5.~ P pw~rful ~dees 
There is little doubt that we will be unable to 
solve the problems of coverage in our school 
and university subjects and of predicting whet 
specific knowledge our students will need in 
thirty or forty years. Despite the fact that we 
would like to have more empirical evidence that 
problem solving skills can be taught, we have 
little choice, because we don't have any real 
alternative (for a detailed discussion of this 
issue, Bee SIMON 1978). 
Cognitive Science and Artificial Intelligence 
have contributed to our understanding of 
problem solving processes and we believe that 
general problem solving skills, crystallized as 
powerful ideas, can be taught explicitly in the 
context of a rich environment of problems. The 
main goal of this paper is to show that CL 
provides this rich context (which if it is not 
superior than mathematics, st least complements 
mathematics). 
Powerful ideas are nuggets of knowledge, which 
ere universally useful, which appear over and 
over in different disciplines and which can be 
connected In • natural and illuminating way 
with a large complex of other ideas. 
One example of e powerful ides is the 
heuristic: "divide and conquer". It appears to 
be an almost universal truth, but how it ia 
done in the context of a concrete problem 
situation is far from being trivial. Many of 
the typical problems (like writing a program to 
compute Factorial, to sort • set of objects or 
to solve e trivial puzzle) are too simple, so 
there ia llttle need to use this heuristic. 
Furthermore many traditional programming 
systems are not build for (or do not even 
support) this problem solving approach, whereas 
in our work the heuristic took on a concrete 
meaning and was the only successful way to 
solve a problem. 
In the following parts or this section we 
briefly describe a sat or powerful ideas which 
can be explored in the context of realistic 
problems and research areas in CL (the projects 
are fully described in BOECKER/FISCHER, 1978): 
11 difference between syntax end semantic (eg 
in the context of writing a program to generate 
poetry, in solving word problems in algebra) 
2) rule systems (s sort of production systems; 
they ere useful for the organisstion of 
programs to derive the plural form of an 
English noun given the singular, to conjugate 
French verbs, to characterize the rules and 
heuristics in a game or to implement the 
evaluation procedure of LISP) 
31 design and Implementation of s mini-language 
(this silo.as us to describe s problem in terms 
which ere characteristic for the problem 
instead of being forced to use e given general 
purpose programming language; examples ere: 
production systems, pattern matcher, etc); in 
programming, it is a natural task to design 
your own representational system whereas in 
mathematics people mostly use the 
representational system given to them 
4) experimentation with a wide variety of 
different grsmmewa (eg to generate and analyse 
algebraic expressions; to transform arabic 
112 
numerals into roman ones and vice versa; to 
explore transition networks and augmented 
transition networks in dealing with natural language) 
5) knowledge representation: eg to derive 
implicit knowledge and to study the impact of 
processing at read-time (antecendent theorems) 
versus question-time (consequent theorems) in a 
system which dealt with family relations (a 
system of this sort can be contrasted with 
ELIZA or a program to cast horoscopes); the 
following diagram illustrates how ? implicit 
relationships (---) can be derived from 3 
explicit ones within a family of four persons: 
', ~/ ~,~ "\ |1 ", \,\~, "~-~ 
"-- l I"o ' 
6) exploration of statistical properties of 
languages in the context of s wide variety of 
different language games (eg like designing the 
distribution of a Scrabble game, information 
content of vowels in different languages etc) 
7) general computational ideas (eg like 
backtracking, which is encounterd in parsing 
non-deterministc grammars and which could be 
applied to pattern matching and tree like data 
structures) 
6. Pattern Matchinq- an example for the.deslqn 
9nd implementation of s minirlsnquaqe 
A matching capability can be a key element for 
many problem solving tasks involving the 
computer to make otherwise large, complicated 
efforts reachable. The following powerful ideas 
can be investigated in the context of this 
project: 
I) incremental design: we can start with s 
pattern marcher which is basicly en EQUAL 
predicate. The next steps could be: a 
membership predicate, s pattern with slots of 
fixed size, s pattern with slots of arbitrary 
size (which creates the need for back-up), 
the possibilty for simultanous assignment of 
matched elements to pattern variables, the 
restriction of matching by using predicates 
etc 
Z) the problem is ill-defineds the 
specification of the pattern marcher should 
be derived from the needs of using it to 
simplify problem solving tasks. A partial 
implementation can be an important help for a 
further specification or for a revision of 
already existing parts, ie the problem 
formulation is an important part of the 
problem solving process 
3) definition of a new language layer: the 
pattern matcher can be used as a new language 
layer between the problem and the programming 
language and it can help to reduce the 
distance between the two. 
4) glass-box approach: in many situations, we 
are primarily interested in using the pattern 
marcher. But by making use of an already 
existing program the student is not confined 
to a black box (like it would be in CAI 
environment); at any time he/she can look 
inside the program, open it up, change it to 
his/her own needs etc. A prerequisite for a 
program to be a glass-box is that it is 
implemented in a formalism the student is 
familiar with. 
5) recursive control structure, a pattern 
marcher is a good example for the power of 
recursive definitions and control structures 
which can be used in many other situations 
A pattern marcher can be used in all projects 
where symbolic structures have to be dissected 
and identified, eg for the translation from 
infix to prefix, for parsing and translating 
processes, for morphological analysis, for 
simple I/0 routines (eg the identification of 
keywords), for ELIZA like programs and for 
symbolic manipulation of algebraic expressions. 
We do not have the space to document the 
problem solving processes (including all the 
incomplete versions) which "occured in the 
context of implementing the pattern marcher 
(see BOECKER/FISCHER 1978) but we want to give 
examples of its use. The simplification with 
the help of a pattern marcher can be 
demonstrated by a program for infix to prefix 
translation (written in LOGO! the program also 
nicely shows the power of recursive 
definitions): 
~0 PRM"ZX : IIIVIX 
10 LOCAL "A "B 
20 IF (EQUAL COUNT :INFIX I) TH\]~ 0UTFJT :INFIX 
3o zF ~ \[?A + ?B\] :inFiX 
OVrIK~ (s~?l~c~ "SUM I~IX :A PREFIX :B) 
OUTPUT (S~NT~CE "DI~I~/~CE PREFIX :A PREFIX :S) 
50 zF MA~ \[?A * ?B\] :n~zx 
~ (S~TI'~CE "PRODUCT ~IX :A PREFIX :B) 
60 ~ ~m \[?A I ?S\] :zm~zx 
OU'I'I~ (S~I'I'~OE "QUOTIENT PREFIX :A PREFIX :B) 
70 == \[~Om S~AX\] 
The following testruns show how the program 
works: 
su~u v 
113 
A ~l~E~ ~o~r ~ ~o~ c A 
This version of the program can be extended 
easily to include other operators like ">" or "<" 
: 
~ (S~'I~C@ "~'l"~ ~ :~ R~I~ :~) 
CIJTRE (S~TE~CE "I~ I~IX :a PREFIX :~) 
It ie an instance in the class of rule systems 
which we mentioned earlier. The ordering of the 
rules takes care for the precedence conventions 
of infix notation. We have chosen this 
application specifically to support our claim 
that many problems considered to be 
mathematical can be more clearly understood by 
looking st them from • linguistic viewpoint 
(and the APL experience shows that changing the 
precedence rules for the evaluation of 
arithmetic expressions poses a non-trivial 
problem). 
Another application of the pattern marcher 
would be to parse sentences in a language where 
the grammar is given. For this purpose we 
assume that the pattern may contain predicates 
(which ere marked by "<" and ">"): 
TRUE 
The following grammer may serve ss an example 
(it describes the language of st least one "O" 
followed by at least one "1"): 
<$8~1~ --> <SO> <$1> 
<SB> --> 0 I 0<$1> 
<51> --> 11 1<51> 
SENT, SO and SI can be implemented with the 
pattern mstcher aa lollowed: 
Io 0OTI~ ~TCHP \[<SO> <sl>\] :I~t~ 
~OSO :D~er 
10 ~Y ~tITC\]~ 0 :\]\];POT ~ ~ "~ 
2o 0mg~ ~nc~ \[o <so>\] :n~oT 
~0 $I :I\]l@~ 
10 :IY ~t~TC\]~ I :D~t~ ~ Otfigb5 "TI~ 
20 OU'lg~ ~ \[:1 <St>\] :~ 
A few testruns show the working of the parser: 
?ALSE 
I~II~ ~ \[0 0 0 1\] PIlINT SI~T \[0 t O 1\] 
TRUE FALSE 
7. Imolicet~ons for problem eolv~nq and 
education 
Powerful ideas have the potential to lead to a 
breakdown of the traditional boundaries between 
established scientific disciplines and reduce 
the division of school knowledge into 
disJunctive compartments. ~By working on some of 
the proJects described above our students found 
that the knowledge which they acquired or 
discovered was not only useful in the context 
of a specific task but could be successfully 
used to understand end solve problems in other 
domalns as well, which should be illustrated 
through the following two specific examples: 
1) the students became aware that the 
evaluation of arithmetic expressions (as it 
is commonly used in mathematics) is not 
something determined by God but that it is 
only s convention and that the laws behind it 
can be easily explained by the use of a 
grammar. 
2) s student discovered why mathematicians 
talk about one-to-one mappings (whlch never 
made any sense to him in mathematics) by 
trying to design secret codes in some of the 
language games (eg Pig Latin and other ones) 
Another important feature of our approach wee 
that the students extended the range of their 
"subjectively computable" problems, which 
helped them to replace their view of the 
computer being a giant adding machine with the 
more adequate view of being s general 
information processing device. We challenged 
their views thinking about the computer. 
Despite the fact that computation is still in 
its infancy there are many strong beliefs whet 
computers are, whet they can do and what they 
can not do. 
By being exposed to the complex problems 
mentioned above the students got familiar with 
general problem solving ideas about 
representations, planning and debugging. The 
intuitive understanding which a person has 
about his/her own language provided the basis 
that debugging incomplete and incorrect 
programs becomes an easy-to-grasp activity, 
because bugs in language programs have a high 
visibility (ie we can discover them by 
inspection and not only by extensive 
calculations like it is the case in numerical 
computations). 
Problems in CL provide good prototypes to 
understand the theoretical relevance of 
debugging. Opposed to the dominant view in 
114 
computer science, where many people regard bugs 
as an awkward obstacle (or as an indication 
that the programmer is unable to think clearly 
and carefully enough) we consider bugs as 
potentially informative friends and as a 
starting point to find out about the 
discrepancies between'a specification (a model, 
e theory) and an implementation (s program). In 
CL, most people are aware that if a conflict 
arises we can not always conclude that the 
specificat.ions are correct and the 
implementation is wrong (as in Galileo's case, 
where the theory was wrong and his data were 
correct). 
Working on the projects described above, the 
students can do work which is close to the 
research front (if they would have done their 
work ten years earlier they could have earned e 
PhO degree with it). This makes this subject 
material once again more interesting than much 
of mathematics where the students have to think 
about what is not even close to the current 
research front. 
9. \[mpiricel findinqe 
Most of the hypotheses and assertions of the 
previous sections ere supported by the 
empirical work in our project. We have not made 
an effort to do any kind of formal evaluation, 
but we have carried out a large number of 
informal investigations to understand the 
impact of our approach. Students filled out 
questionaires, participated in think-aloud 
protocols for many problem solving situations 
end we tried to understand their programs and 
the bugs they produced during the solution of a 
complex problem. There is no space here to talk 
about this in detail; the information is 
documented in KLING et al (1977) end FISCHER 
(1978 end 1979). 
We believe that our approach turned out to be 
very successful. The students enjoyed working 
in our laboratory and they learned a lot about 
language as well as general problem solving snd 
programming skills. Especially students with 
little interest in mathematical problems were 
motivated by language-oriented applications. 
They could work in an active mode and 
investigate arbitrary formalisms and 
conjectures. They could see that ideas from 
linguistics could help them to understand 
problems in other domains, which supports our 
hypothesis that problems from CL can serve as 
an entry point and as a transient object to the 
world of problem solving, programming end 
mathematics. 
Acknowledqements 
I would like to thank H.-D. Boecker, A. Fauser, 
3. Laubsch end O. Roesner for many critical 
comments about earlier drafts of this paper. 
References 
Boecker,H.-O. and G. Fischer (1978): "Interaktives 
Problemloesen mit Computerhilfe: Problemaufgaben zur 
Linguistik, Informetik und Kuenstlichen Intelligenz", 
Forschungsg~CUU, Darmstadt 
Fischer, G. (1978). "Probleme und Erfehrungen bei dec 
Programmiersuabildung im Informatik-Unterricht" in W. Arlt 
(ed) ; "EDV-Einsatz in Schule und Ausbildung", Oldenburg 
Verlaq, Huenchen, pp ?0-75 
Fischer, G. (1979). "Fehlerdiegnose - Grundbauatein fuer ein 
Verstehen yon Lehr- und Lernprozessen", in Beitreege zum 
Methematikunterricht, Soh~oedel Verleg 
Key, A. (1977): "Microelectronics and the personal computer", 
Scientific America 1977, pp 2}I-2~w~ 
Kling, U., Boecker H.-D., Fischer, G., Freiburg, D., 
Schneider, B. end Schroeder, 3. (1977): "Projekt PROKOP", 
Forschongscjruppe CUU, Darmstadt 
Papert, S. (1979): "The LOGO Book", unpublished draft, MIT AI 
Lab 
Simon, H. (1978): "problem Solving and Education", CIP Working 
Paper No. }91, Carnegie Mellon University 
115 

