A Corpus-Based Grammar Tutor 
for Education in Language and Speech Technology 
Lars Borin and Mats Dahll6f 
Department of Linguistics, Uppsala University, 
Box 527, SE-751 20 UPPSALA, SWEDEN 
E-mail: Lars. Borin~ling. uu. se, Nats. Dahllof@ling. uu. se 
Abstract 
We describe work in progress on a 
corpus-based tutoring system for edu- 
cat, ion in traditional and formal gram- 
mar. It is mainly intended for lan- 
guage and speech technology students 
and gives them the opportunity to learn 
grammar and grammatical analysis from 
authentic language material. The ex- 
ercises offered by the system are based 
on pedagogically adapted versions of for- 
malisms and tools that are likely to be 
of relevance to the students also later in 
their professional life. The system will 
be continuously evaluated in university- 
level courses, both in order to assess its 
effectiveness as a learning aid and to pro- 
vide guidance in its further development. 
1 Background 
In this paper, we describe work in progress on 
a corpus-based grammar tutor. The inspiration 
comes from the authors' extensive experience of 
teaching traditional and formal grammar to stu- 
dents specialising in computational linguistics and 
language engineering or--to use the terminology 
adopted for this workshop--Language and Speech 
Technology (LST). This practical teaching expe- 
rience, from both elementary and advanced uni- 
versity courses, provided the original impetus to 
undertake the work described here. In particu- 
lar, we were motivated by the following consider- 
ations. We wished: 
• to raise the generally poor level of grammar 
skills in our students, in an economic reality 
where tile resources to provide individual in- 
struction are conspicuously lacking; 
• to base the grammar learning on realistic 
data. both as to the kind of language anal- 
ysed a.nd as to the formalism used; 
• to separate for didactical purposes two con- 
ceptually, but not practically, independent 
components of a natural language grammat- 
ical description, viz. grammar and lexicon, 
without reducing either component to trivi- 
ality; 
• to go from the simple to the complex in terms 
of the expressiveness of the formalism; 
• to support a hypothetico-deductive learning 
style "learning by inquiry" (McArthur et al. 
1995) based on intrinsic feedback (Laurillard 
1996). 
1.1 LST students' grammar skills 
Grammar, which used to play a leading role in 
teaching and learning languages, has been rel- 
egated to a fairly subordinate position in con- 
temporary language pedagogy. Instead, lan- 
guage learning theorists currently favour so-called 
communicative approaches to language teaching. 
While it is true that most researchers in the field 
of second language learning (e.g. Ellis 1985; Light- 
bown and Spada 1993) recognise learning situa- 
tions and learner types for which grammar may 
prove the most effective means of learning at least 
some aspects of a second or foreign language (see, 
e.g., Batstone 1994), even here it is not always 
explicit grammatical reasoning that the authors 
have in mind (see Underwood 1984). Theoretical 
insight in grammar is, on the whole, not regarded 
as very useful for language learners. 
We do not wish to take issue with the stand- 
point that foreign or second languages in general 
are better learned through conversational interac- 
tion than through the study of grammar. It is 
however obvious that the subject (both morphol- 
ogy and syntax) is a matter of vital importance 
for LST (and linguistics) students. They con- 
sequently need a solid grasp of the fundamental 
grammatical and lexicological concepts and the 
ability to apply them to the analysis of texts. 
They also need to be able to use grammar for- 
malisms to state general grammatical principles. 
From our point of view, the ideological shift in 
language pedagggy has brought with it the unfor- 
tunate consequence that the students who enroll 
on our courses generally know less grammar than 
used to be the case. This in turn has forced our 
department to ~estructure both the language engi- 
neering and general linguistics curricula, through 
the addition of courses covering elementary gram- 
matical concepts and techniques, knowledge of 
which we earlier could take for granted in our stu- 
dents. 
Grammatica! analysis is a skill which--like 
many other sldlls--is best acquired and honed 
through its application to concrete reality, e.g. to 
words and sentences as found in authentic con- 
texts, and by :writing grammatical descriptions 
of (fragments of) natural language. At the same 
time, practical exercise sessions with a tutor are 
among the more costly modes of teaching. This is 
a pedagogical problem in times when many edu- 
cational institutions experience financial cutbacks 
and the number of teacher hours per student de- 
crease. 
Much would thus be gained if we could offer 
the students good self-study materials for gram- 
mar practice, good in the sense that they would 
be pedagogically sound, but also in the sense that 
they actually would be used by the students. Be- 
cause of this, we are interested in investigating 
how computer-based grammar instruction mate- 
rial should be designed. For theoretical inspira- 
tion we have ~urned to the findings of the re- 
search in Computer-Assisted Language Learning 
(CALL). The reasons for this is that both CALL 
and our prese~!t aims are about computer-based 
language-related training and that CALL is a 
large and growing research area. 1 And, indeed, 
as we will se below, many of the insights of the 
CALL community seem to be directly relevant to 
the case at hand. 
2 Pedagogical considerations 
There are some pedagogical points that we wish to 
raise in connection with the design of a grammar 
tutoring system ibr LST students, and which we 
feel are inadequately addressed in existing systems 
of this kind. 
'This connection is natural to us also because we 
offer CALL as one of the specialisations in language 
engineering. 
2.1 The importance of authenticity 
Several pedagogical systems support training in 
formal grammar writing (Gazdar and Mellish 
1989; Antworth 1990; von Klopp and Dalton 1996; 
McConnel 1995; Beskow et al. 1997; see also 
Rogers 1998). In most cases these systems only 
deal with grammars from an abstract point of 
view, without calling attention to the issue how 
well a grammar accounts for real language. These 
systems do however offer the students valuable fa- 
cilities, e.g. allow them to evaluate a grammar by 
using it to parse arbitrary strings or for random 
generation. For our purposes, these systems are 
"realistic" in one sense, namely in that they let 
students express linguistic generalisations in for- 
malisms which are similar to those actually used 
by language technologists. 
In another sense, however, systems of this kind 
are spiritually kindred to the "intuitive" method 
in generative grammar, rather than to the goals of 
language engineering. The issue of how relevant 
data is to be found and used is normally left out of 
the picture altogether. This is a major pedagog- 
ical defect as the step from understanding gram- 
mars as formal systems to understanding them as 
theories about existing language use is both cru- 
cial and intellectually demanding. It is our expe- 
rience that this is one of the most difficult aspects 
of education in formal grammar. We consequently 
think that there is much to gain by the use of a 
tutoring system that helps the student to see how 
a grammar relates to a morphosyntactically anno- 
tated corpus. The aim of the work described here 
is to develop a system which will introduce gram- 
mar writing as an empirical process with the aim 
of accounting for authentic language. 
2.2 Divide and conquer 
The use of a tagged corpus as a testing ground for 
fledgling formal grammar writers confers another 
advantage which is often absent from the systems 
referred to above. Since the aim of these systems 
is to train the students in writing syntactic or mor- 
phological rules, the lexicon is more often than not 
reduced to the absolute minimum--both in the 
number and in the complexity of entries--needed 
to illustrate how the syntactic or morphological 
rule system works. This is indeed a problem, but 
it can not be solved simply by urging the students 
to compile extensive lexicons. On the contrary, 
there is a clear pedagogical point to the separation 
of the grammar from the lexicon for training pur- 
poses. Generally, it is a good principle to present 
new material a little at a time, in conceptually 
coherent portions. Otherwise, the students may 
get confused, and as a consequence frustrated. In 
this case, you would like to offer them a ready- 
made lexicon which should be flexible enough to 
accommodate a number of grammar formalisms (a 
"poly-theoretic" lexicon). A morphosyntactically 
tagged corpus can be made to stand in for such a 
lexicon, at least in some respects; in addition to 
the purely linguistic information contained in it, 
there is also (implicit) information about frequen- 
cies of occurence in authentic language, about col- 
locations, etc. Even if there is lexical information 
which will not, as a rule, be found even in a fairly 
richly annotated corpus (e.g. valency information 
and semantics), the information that you can find 
there still constitutes a vast improvement over the 
typical lexicons of grammar training systems. 
Conversely, the tagged corpus makes an excel- 
lent basis for exercises aiming at learning to iden- 
tif:y the "atoms" of grammar, i.e. parts of speech 
and inflectional categories, in a realistic context. 
There are some tutoring systems for this purpose 
(e.g. Qiao 1996), including one (Mats 1999) that 
we have been trying out in our department re- 
cently. McEnery et al. (1995) compare another 
such system (the one described by McEnery et 
al. 1997) to traditional human teaching in a con- 
trolled evaluation procedure, and reach the con- 
clusion that tile corpus-based computer-assisted 
method yields slightly better learning results. 
2.3 First things first 
It, is a good pedagogical principle not only to di- 
vide that which is to be learned into manageable 
chunks, but also to proceed from simpler to more 
complex knowledge. Ideally, the tutoring program 
should impose exactly this ordering for those stu- 
dents that need it (see Lanrillard 1996). The mor- 
phosyntacially annotated corpus puts at the stu- 
dents' disposal a "lexicon" which will tag along, 
as it were, as 
• they learn to identify not only which part of 
speech a certain text word is, but also which 
inflectional information should be associated 
with it; 
• their grammars evolve in terminal complex- 
ity from simple phrase structure rules with 
atomic terminal categories, to unification- 
based grammars with feature structures en- 
coding the full morphosyntactic information 
for each lexical unit; 
• their grammars evolve in nonterminal com- 
plexity, enabling them to analyse increasingly 
larger portions of the corpus. 
2.4 Learning by inquiry 
A corpus-based grammar tutor shares with 
corpus-based CALL in general the trait of be- 
ing eminently suited for hypothetico-deductive, 
problem- and data-driven learning ("serendipity 
learning"; cf. Flowerdew 1996, or "learning by 
inquiry"; see McArthur et al. 1995). By work- 
ing with the program the student will develop his 
skills in evaluating a grammar as an account of 
the syntactic phenomena found in a corpus. The 
system will support a process of thinking that 
highlights important aspects of scientific reason- 
ing. Abstract concepts such as theory, data, pre- 
cision, recall and prediction are illustrated in a 
fairly concrete manner, as are (other) basic as- 
pects of formal grammar. 
3 The grammar tutor 
With these aims in mind, we are developing 
a corpus-based grammar tutoring system. We 
are aiming at first for a system with limited 
functionality--both in order not to overreach our- 
selves and to facilitate evaluation--which will un- 
dergo several rounds of formative evaluation (see 
Laurillard 1996). 
The system will provide a learning context that 
in important respects is a realistic one. The stu- 
dents will work with authentic linguistic material, 
in the form of a tagged corpus, and use peda- 
gogically adapted versions of formalisms and tools 
that they will be using also later in their profes- 
sional life. This is similar in spirit to the approach 
taken by McArthur et al. (1995), who argue per- 
suasively for the use in education of so-called ES- 
SCOTS (Educational Support Systems based on 
Commercial-Off-The-Shelf software). They report 
both an unusually short system development time 
and good learning results (in an experiment where 
they adapted a commercial Geographic Intbrma- 
tion System (GIS) for use in an educational set- 
ting). 
The system will be used and evaluated in the 
context of one or more of our LST and linguistics 
courses (formal syntax and computational syntax, 
at least, possibly also basics of grammar), starting 
in the autumn term of 1999. The evaluation will 
not be carried out as a test group-control group 
setup. This is mainly for practical reasons, our 
student population being too small for this kind of 
experiment. 2 Instead, we will use in-class observa- 
2There are also theoretical motivations for this, as 
there have been serious concerns voiced in the litera- 
ture about the meaningfulness of such "experiments" 
in the context of computer-assisted learning (see Borin 
1998). 
I 
tion, questionnaires and interviews with the stu- 
dents and teachers, and logging of student activ- 
ity as our main evaluation instruments. The eval- 
uations will, hopefully, yield two kinds of result. 
Firstly, we expect to learn something about the ef- 
fectiveness of using a corpus-based computerised 
grammar tutor i and, secondly, we will see what 
should be changed and what added in the system 
(this is what the "formative" part is about). 
3.1 Corpus and exercise types 
As just stated, any annotated corpus could form 
the basis of the grammar tutor. As our point \] 
of departure, we have chosen to use a Swedish 
one-million-woad balanced corpus, the Stockholm 
U'me,~ Corpus (SUC; Ejerhed and K~llgren 1997). 3 
For the first version, there are two grammar ex- 
ercise types under development: The most basic 
exercise is to assign part of speech and morphosyn- 
tactic features to words in the corpus. This exer- 
cise exists in a preliminary version (Mats 1999), 
which has been used at our department with en- 
couraging results. 4 
The second step is the formulation of grammat- 
ical rules and applying them to the corpus with 
the help of a built-in parser. Random analysis 
and generation with the same grammar will also 
be supported. The system will eventually support 
two formalisms, plain context-free grammar and a 
feature-structure based one. 
The parser helps the student to evaluate his/her 
grammar by making clear which analyses the 
grammar assigns or fails to assign to the sub- 
strings of the corpus. One grammatical category 
(non-terminal symbol) is selected as the one be- 
ing of particular interest for the moment. The 
parser locates all strings that are generated as in- 
stances of that category. The corpus provides the 
lexical nodes, i.e. the text word-category pairings. 
By inspecting these analyses the student will be 
in a position to decide, with respect to a certain 
:tSUC was compiled and semi-automatically tagged 
m the years 1989-1996 (Ejerhed and K£11gren 1997). 
The corpus follows the Brown Corpus format: There 
are 5(}0 text chunks of approximately 2000 words each, 
with a genre distribution similar to that in other bal- 
anced corpora, although only the written standard 
language is represented. A corrected second version 
of the corpus is due to appear before the end of 1999. 
'~It has been tried out during the spring term of 
1999 with a group of computer science students tak- 
ing a course in language engineering in our depart- 
meat (Mats 1999). The students were largely positive 
in their evaluation of the exercises, but they also sug- 
gested some improvements in the user interface and in 
the way the material was presented to the user. We 
will incorporate some of these suggestions in the next 
version of the exercise. 
category, to what extent the grammar accounts 
for the instances of the category and to what ex- 
tent it overgenerates. The tokens found may be 
listed (with context) or graphically indicated in 
the running corpus text. This exercise will encour- 
age the student to evaluate a grammar in terms of 
its precision and recall with respect to the selected 
category. The student's own grammar-related in- 
tuitions are, of course, important in this kind of 
corpus-oriented setting, as only the words of the 
corpus are tagged (it is not a treebank). In other 
words, there is no predefined right answer avail- 
able (but see below). The evaluation of the stu- 
dent's performance is rather based on his/her own 
judgments. This is an example of so-called intrin- 
sic/cedback, which is the best kind of feedback, 
according to several CALL practitioners; see Lau- 
rillard 1996. Nevertheless, the system will ensure 
that the application of these intuitions and the 
reasoning about the grammar will be supported 
by considerations of concrete data. 
The tagged corpus may also be used for random 
generation. The text word-category pairings de- 
fine a lexicon which generates expressions of var- 
ious categories in conjunction with the student's 
grammar. In this way the lexical material of the 
corpus and the grammar are used to make gram- 
maticality predictions. The generation exercise 
will mainly throw light upon how overgeneration 
problems are discovered and dealt with. 
The system gives some feedback about the sta- 
tus of the grammar. Warnings are issued if some 
category is left undefined. The number of rules, 
categories, and features used is also reported. 
This is intended to alert the student to the issue 
of how simple/complicated the grammar is, which 
is important as simplicity is one of the most im- 
portant aspects of theoretical adequacy. 
3.2 Parts of speech and grammars 
The tagging provides the link between the stu- 
dent's grammar and the given corpus data. It is 
therefore crucial which categories are used. As the 
empirical material is a selection from a particular 
corpus, the tags visible to the student must be 
derivable from the tagset used in that corpus. Of 
course, these tags may be mapped onto the tags 
of the tutoring system in various ways. 
The system comes with two predefined map- 
pings from corpus tags to grammar categories, to 
context-free categories on the one hand and to fea- 
ture structures on the other. These mappings are 
defined in a file and can be revised by the teacher. 
Manipulation of this mapping can, of course, also 
be a part of more advanced exercises for the stu- 
dent. 
As mentioned, the system will support two 
grammar fbrmalisms--corresponding to the two 
tagset mappings just mentioned--pure context- 
free grammar and a feature-structure formalism, 
the latter in the style of PATR-II (Shieber 1986). 
A context-free grammar is (by definition) used 
together with a fiat taxonomy of lexical categories. 
As the default option, the program operates with 
such an inventory of categories which is related to 
the traditional part-of-speech system, but more 
fine-grained. 
The feature structure tags used with the PATR- 
II-style formalism correspond to the full informa- 
tion in the tagset used, i.e., they contain primarily 
inflectional information, in addition to the syntac- 
tic category. This means that the corpus mainly 
will support constraint-based accounts of agree- 
ment phenomena. However, the system as such 
will allow descriptions dealing with arbitrary as- 
pects of grammar. 
3.3 Implementation 
As the implementation language we have cho- 
sen Java, primarily because of its platform- 
independence and because it is an excellent lan- 
guage for rapid prototyping of applications with 
sophisticated GUIs, but also to some extent be- 
cause of its association with the Internet and the 
WWW (see below). 
3.4 Planned developments 
Explicit evaluation of the students' actual use of 
the system will, needless to say, provide the main 
indication of how the system should be improved 
and extended. The implemented exercises have 
nevertheless been designed to fit into a scheme 
of logically linked exercises, which step-wise lead 
the students on to more complicated and difficult 
tasks. 
The present system could in a natural way be 
extended to deal with a corpus which is preanal- 
ysed also with regard to constituent structure. A 
less advanced task for the student would then be 
to write a grammar that agrees with the given 
structure(s). The system would provide detailed 
feedback evaluating the ability of the grammar to 
generate the given syntactic structures. This ex- 
ercise would illustrate the purely formal aspect of 
grammar tbrmulation. It could preferably be used 
as a preparation for the exercises relying on intrin- 
sic feedback from the student's own grammatical 
intuitions. 
Another valuable addition to the system would 
be a module that encourages the student to organ- 
ise the empirical evaluation in a systematic way. 
The compilation and use of test suites provide an 
often used and simple method with this advan- 
tage. A test suite for a certain category is a list 
of known instances of the category and a list of 
strings that are known not to belong to the cat- 
egory. A test suite thus provides a collection of 
data against which a grammar may be automati- 
cally evaluated. The system reports the number of 
positive instances the grammar fails to account for 
and the number of overgenerations. This exercise 
shows how the empirical evaluation of a grammar 
may proceed in a more systematic fashion and en- 
courages trial and error experimentation with the 
grammar formulation. 
Another dimension of difficulty is given by the 
two grammar formalisms. The basic idea is that 
the system should be a pedagogically organised 
toolbox for grammar formulation and corpus in- 
spection (taking the ideas presented in Lager 1995 
one step further) and this idea makes it natu- 
ral to integrate various extensions into the sys- 
tem, such as new inspection tools and other gram- 
mar formalisms and parsers, e.g. that described 
in DahllSf 1999 or finite-state formalisms for syn- 
tax (e.g. Karlsson et al. 1994) or morphology (e.g. 
Karttunen 1993). 
In the context of feature-structure grammars, 
a unification-failure explanation generator is use- 
ful. This component indicates which feature mis- 
match(es) made it impossible for the grammar 
rules to assemble a certain phrase. A simple 
version of this facility is implemented in DahllSf 
(1999) and it has turned out to be very useful 
during grammar construction. Pedagogically de- 
veloped versions of it would likely be valuable for 
students (and professionals) as it often is very 
difficult to see how feature~assignments interact 
in a constraint-based grammar and to locate the 
source of unwanted unification failures. 
A longer-term goal would be to provide the sys- 
tem with intelligent error analysis and help facili- 
ties. This is an exciting but largely unexplored re- 
search topic in CALL, known as Intelligent CALL, 
or ICALL, which draws on research in the fields 
of Artificial Intelligence and Computational Lin- 
guistics. 
It would also be desirable to develop some kind 
of authoring interface to the system. Direct ma- 
nipulation of the system's Java code would pre- 
suppose fairly advanced programming skills and 
this would presumably make it impossible fbr 
most teachers to adapt the system to new learn- 
ing tasks. An authoring facility, allowing users to 
define new exercises in a suitable authoring lan- 
guage, would consequently extend the usefulness 
of the system. Such an interface can also be given 
a more direct pedagogical motivation: There are 
CALL applications where students step into tile 
role of the teacher, as it were, designing exercises 
(as if) for their fellow students, and learning about 
the subject matter in doing so (see Borin 1998). 
In its first version, the grammar tutor will, for 
practical reasons, be accompanied by written in- 
structions and conventional coursebooks. We do 
however intend to integrate this information in the 
system. 5 
3.5 Benefits from Internet use 
As we mentioned above, our choice of Java as 
the programming language for the grammar tu- 
tor was only partially motivated by its status as 
th.e programming language of the World Wide 
Web. Rather, we chose it because it is platform- 
independent and because the GUI capabilities we 
need are built into the language. ~ Thus, the ap- 
plication was not built with the WWW in mind, 
although it is fully feasible to use it over the In- 
ternet. In this case, a possible division of labour 
could be implemented, where the exercise pro- 
grams are Java applets locally executed in the 
student's computer, while the corpus resides in a 
server-side database. 
From the experiences of the CALL community, 
we know that the Internet can bring two distinctly 
different kinds of pedagogical added value to a 
learning situation: 
1. In this case, the pedagogical value is only inci- 
dental upon the general advantage of a client- 
server setUp, i.e. that it is easier to main- 
tain and upgrade an application if you only 
have to do, it once and in one location. For a 
CALL application, this means that data and 
exercises probably can be updated more often 
than otherwise would have been the case. 
2. Tlle other case turns around using the 
Internet as a widely accessible time-of- 
day-indepdndent communications network. 
Thanks to the Internet, students and teach- 
ers, who may be geographically far apart, 
can collaborate both asynchronously and 
synchronously in creating an optimal vir- 
tual learning environment for some types of 
r'This matter may deserve some deliberation. 
Benyon et al. (1997) point out that turning written 
coursebooks directly into hypertext rarely yields good 
results, and m Nygren (1996), on the basis of practi- 
cal experiences of medical information systems, we are 
warned that paper-based information often loses in lu- 
(:idity and navigability as a result of it being poured 
into a computer. 
(~The A~VT and JFC class libraries. 
learning tasks (Pennington 1996; Warschauer 
1996; Levy 1997; Borin 1998). 
The grammar tutoring system has been de- 
signed with self-study in mind, so that it is hard 
to see how it could benefit pedagogically other 
than incidentally--i.e, as in (1) above--from being 
made into an Internet application. On the other 
hand, positive learning effects have been noted 
in situations where students cooperate in fi'ont 
of the computer to do the exercises in a CALL 
program designed for self-study (Chapelle et al. 
1996). This points to the possibility of designing 
for a more central role of the Internet even in a 
program such as the one discussed here. Thus, for 
instance, grammatical analysis could be carried 
out collaboratively (or competitively) by several 
students over the network. 
4 Conclusions and future prospects 
Summing up, we propose to let LST students learn 
grammatical analysis and formal grammar writ- 
ing by practicing these skills with the help of a 
tutoring system which provides a learning envi- 
ronment which in many ways is a realistic one, 
both as regards data and formal methods. The 
grammar exercises will have an empirical connec- 
tion to authentic language in the form of a tagged 
corpus; the formalisms and tools will be of the 
same kind as those used in "real-life" LST; and 
grammar formulation will be presented as a case 
of hypothetico-deductive problem solving. 
The pedagogical adaptation consists, as of now, 
in the following: (1) arranging the learning situa- 
tion so that the students' practice at each moment 
is focussed on one component of the subject which 
is to be learned, while keeping the other compo- 
nents as realistic as possible; (2) going from tile 
simple to the complex; (3) making sure that there 
is adequate feedback (preferrably intrinsic) at all 
times. Further, we plan to evaluate the grammar 
tutor continuously in actual LST and linguistics 
courses in our department, both in order to assess 
its effectiveness as a learning aid and to give us 
guidance in its further development. 

References 

Antworth, Evan L. 1990. PC-KIMMO: A two- 
level processor for morphological analysis. 
Occasional Publications in Academic Corn- 
pitting 16. Dallas: Summer Institute of Lin- 
guistics. 

Batstone, Rob 1994. Grammar. Oxford: Ox- 
ford University Press. 

Benyon, David, Debbie Stone and Mark 
Woodroffe 1997. Experience with develop- 
ing multimedia courseware for the World 
Wide Web: the need for better tools and 
clear pedagogy. International Journal of 
Human-Computer Studies, 47 (1), 197-218. 

Beskow, Bj6rn, Torbj6rn Lager and Joakim 
Nivre 1997. Linguistic Instruments: Gram- 
mar Laboratories for the Macintosh. 
http ://www. ling. gu. se/~li/. 

Borin, Lars 1998. Datorst6dd spr~tkinl~irning. 
Dept. of Linguistics, Uppsala University. 
MS. 

Chapelle, Carol, Joan Jamieson and Yuhsoon 
Park 1996. Second language classroom re- 
search traditions: How does CALL fit? In 
The Power of CALL, ed. by Martha C. Pen- 
nington. Houston, Texas: Athelstan, 33-53. 

Dahll6f, Mats 1999. Flexible typed feature 
structure grammar. Dept. of Linguistics, 
Uppsala University. MS. 
http ://stp. ling .uu. se/~matsd/ftf sg/. 

Eierhed, Eva and Gunnel KEllgren 1997. Stock- 
holm Ume£ Corpus Version 1.0, SUC 1.0. 
Dept. of Linguistics, Ume£ University. 

Ellis, Rod 1985. Understanding Second Lan- 
guage Acquisition. Oxford: Oxford Univer- 
sity Press. 

Flowerdew, John 1996. Concordancing in lan- 
guage learning. In The Power of CALL, ed. 
by Martha C. Pennington. Houston, Texas: 
Athelstan, 97-113. 

Gazdar, Gerald and Chris Mellish 1989. 
Natural Language Processing in LISP. 
Wokingham: Addison-Wesley. 

Karlsson, Fred, Atro Voutilainen, Juha 
Heikkil'~ and Arto Anttila (eds) 1994. Con- 
straint Grammar: A Language-Independent 
Formalism for Parsing Unrestricted Text. 
Berlin: Mouton de Gruyter. 

Karttunen, Lauri 1993. Finite-State Lexicon 
Compiler. Technical Report ISTL-NLTT- 
1993-04-02. Xerox PARC, Palo Alto, 
California. 

Lager, TorbjSrn 1995. A Logical Approach to 
Computational Corpus Linguistics. Diss. 
Dept. of Linguistics, G6teborg University. 

Laurillard, Diana 1996. The TELL Consortium 
- formative evaluation report, http ://www. 
hull. ac. uk/ct i/f ormeval, doc. 

Levy, Michael 1997. Computer-Assisted Lan- 
guage Learning. Oxford: Clarendon Press. 
Lightbown, Patsy M. and Nina Spada 1993. 
How Languages are Learned. Oxford: Oxford 
University Press. 

Mats, Erik 1999. Spr£kt£get: en webbaserad 
programvara fSr datorstSdd spr£kinl~rning. 
MS. http ://stp. ling. uu. se/~er±km/ 
spraktaget/. 

McArthur, David, Matthew W. Lewis and 
Miriam Bishay 1995. ESSCOTS for learning: 
Transforming commercial software into pow- 
erful educational tools. Journal of Artificial 
Intelligence in Education, 6, 3-34. 

McConnel, Stephen 1995. PC-PATR Reference 
Manual. http ://www. sil. org/pcpatr/ 
manual/pcpatr, html. 

McEnery, Tony, John Paul Baker and Andrew 
Wilson 1995. A statistical analysis of cor- 
pus based computer vs. traditional human 
teaching methods of part of speech analy- 
sis. Computer-Assisted Language Learning, 
8, 259-274. 

McEnery, Tony, John Paul Baker and John 
Hutchinson 1997. A corpus-based gram- 
mar tutor. In Corpus Annotation, ed. by 
Roger Garside, Geoffrey Leech and Anthony 
McEnery. London: Longman, 209-219. 

Nygren, Else 1996. From Paper to Computer 
Screen. Human Information Processing and 
User Interface Design. Uppsala: Acta Uni- 
versitatis Upsaliensis. 

Pennington, Martha C. 1996. The power of the 
computer in language education. In The 
Power of CALL, ed. by Martha Pennington. 
Houston, Texas: Athelstan, 1-14. 

Qiao, Hong Liang 1996. Processing the Lan- 
caster Parsed Corpus as data for a CALL pro- 
gram: The design and implementation of the 
Word Class Drills vl.0. Computer Assisted 
Language Learning, 9, 163-180. 

Rogers, Henry 1998. Education. In Using Com- 
puters in Linguistics. A Practical Guide, ed. 
by John M. Lawler and Helen Aristar Dry. 
London: Routledge, 62-100. 

Shieber, Stuart 1986. An Introduction to 
Unification-Based Approaches to Grammar. 
Lecture Notes No. 4. Stanford, California: 
CSLI. 

Underwood, John H. 1984. Linguistics, Com- 
puters and the Language Teacher. A 
Co'mmu'nicative Approach. Rowley, Mas- 
sachusetts: Newbury House Publishers. 

von Klopp, Ana and Chris Dalton 1996. 
Interactive teaching material for the World 
Wide Web: A linguistics tutor written in 
J~tva. Department of Linguistics Research 
Reports UWB-LING/RR96-01. University 
of Wales, Bangor. 
http ://www. bangor, ac. uk/ling/staf f / 
avk/icalm, html 

Warschauer~ Mark 1996. Computer-assisted 
language learning: An introduction. In 
Multimedia Language Teaching, ed. by S. 
Fotos. Tokyo: Logos International, 3-20. 
WWW documents as accessed on April 19th, 
1999. 
