I 
A software for language education and rehabilitation 
of autistic-like children 
Elisabeth Godbert, Pascal Mouret, Robert Pasero, Monique Rolbert 
Laboratoire d'Informatique de Marseille 
CNRS and Universit~ de la M~diterran~e 
163 Avenue de Luminy, Case 901 
13288 Marseille Cedex 9, France 
{godbert, mouret, pasero, rolbert }@1 im. univ-mrs, fr 
Abstract 
We present the EREL system, a thera- 
peutic software for the education and the 
rehabilitation of children suffering from 
language disorders, especially devoted to 
"autistic-like" children. This system is 
based on the generic ILLICO software, a 
natural language system for analysing and 
synthesising sentences, and guiding if nec- 
essary their composition. ILLICO com- 
bines two principles: modularity in the rep- 
resentation of knowledge defined at the dif- 
ferent levels of language processing, and 
sentence composition using partial synthe- 
sis and guided composition. We first de- 
scribe the ILLICO system, and show how 
its two characteristics are relevant for the 
development of communication aids. Then 
we describe the functionality of the EREL 
system. Finally, we go into the details of 
an activity proposed by the software, and 
describe some specific elements of NLP re- 
quired for its development. 
1 Introduction 
We describe here the EREL system, a therapeu- 
tic software devoted to the Education, Rehabilita- 
tion and Evaluation of Language devoted to children 
suffering from language (and then communication) 
disorders, and especially devoted to "autistic-like" 
children. In addition to their difficulties to express 
themselves verbally, many of these children suffer 
also from cognitive and/or motor disorders which 
make graphic or written expression difficult, and of- 
ten make these persons keep socially apart (Ornitz, 
1974) (Manrer and Damasio, 1982). But, given a 
motivating context, an individualised surrounding 
environment and materials, some of these children 
may be able to exteriorise capacities which had hith- 
erto remained mute. 
For more than ten years, a large amount of re- 
search has been carried out in the field of communi- 
cation rehabilitation for handicapped persons, and 
technical aids known as "communication aids" have 
now been developed with some success. They can 
use "communication bords containing words (whose 
quantity, i.e. vocabulary, depends on the capacities 
of the user), graphic representations (pictures and 
photos, hierarchies of pictures, e.g. TEACCH pro- 
gram widely used in USA), written communication, 
synthetic voices. They aim to compensate the loss of 
communication for people without the use of speech. 
The use of computers as an intermediary has been 
seen as a natural extension of the use of classical 
communication aids; it may enable to bypass the 
different handicaps temporarily at least, and to com- 
pensate for the problems with language and motric- 
ity (Garoux et al. , 1989) (Howlin, 1989). 
The EREL system, described here, is an applica- 
tion of research in the domain of natural language 
processing and especially of the development of nat- 
ural language interfaces, and has been designed in 
a collaboration with medical staffs (doctors, psy- 
chologists, etc.) specialised in the treatment of chil- 
dren suffering from developmental disorders (such as 
typical and atypical autism, or childhood psychosis) 
(Godbert, 1996). The system provides a set of edu- 
cational play activities illustrated through multime- 
dia technology and designed to stimulate and help 
users to employ common language to express them- 
selves within a specific context. 
The development of EREL is based on the French 
ILLICO system, a generator for various natural lan- 
guage processing systems which provides a set of lan- 
guage processing tools using computational linguis- 
tic techniques making it possible to produce text in a 
"guided mode" (Pasero and Sabatier, 1994) (Pasero 
59 
and Sabatier, 1997). We will show that this mode 
is of great relevance to communication aids for the 
disabled. 
In part 2, we describe the ILLICO system. In part 
3, we mention the state of the art in the domain 
of communication aids for autistic children, and we 
show why ILLICO is relevant to the development of 
software devoted to the rehabilitation of persons suf- 
fering from language disorders. The functionality of 
EREL is described in part 4. Part 5 goes into all the 
details of an activity proposed by the software, and 
describes some specific elements of NLP required for 
its development. 
2 The ILLICO system 
The ILLICO system is a generic system for natu- 
ral language processing (NLP). It provides a set of 
NLP tools making it possible to develop various ap- 
plications, such as intelligent natural language in- 
terfaces for databases, communication aid systems, 
computer assisted teaching or learning systems, etc. 
ILLICO has been designed from the following two 
principles: 
1. Modularity in the representation of knowledge 
defined at the different levels of language pro- 
cessing (lexical, syntactic, semantic, concep- 
tual, contextual levels); 
2. Sentence composition using partial synthesis 
and guided composition. 
Modularity in knowledge representation 
The different types of linguistic knowledge are inde- 
pendently encoded in separate modules; this ensures 
their portability, and makes easier their updates and 
interactions: 
- a lexicon contains expected words and expres- 
sions; 
- a grammar specifies the expected sentence struc- 
tures and the grammatical agreement; 
- a set of semantic composition rules produce se- 
mantic representations from the syntactic rules of 
the grammar; 
- a conceptual model specifies, in terms of rela- 
tions, the world of the application; 
- a contextual model specifies the objects intro- 
duced in the preceding sentences. 
Guided composition of sentences 
The kernel of the ILLICO system carries out an in- 
teractive processing of natural language, based on 
partial synthesis, which checks the well-formedness 
of the produced sentences as the user goes on com- 
posing the sentences. This checking is simultane- 
ously done at all the levels of well-formedness: the 
constraints defined at the different levels are corouo 
tined (i.e. taken into account in a "parallel-like" 
way), in an algorithm which runs either in parsing 
or in synthesis. 
Sentence composition using partial synthesis enables 
the system to offer the possibility to generate sen- 
tences in a "guided mode". In this mode, the user 
is guided while he produces text: at each step of 
the composition of a sentence, the system synthe- 
sises and displays the words and expressions that 
can be used to continue the sentence and that will 
lead to a well-formed sentence. The guided composi- 
tion mode enables the development of user-friendly 
interfaces in which errors on the domain of the appli- 
cation never occur, and in which non-expected (i.e. 
incorrect) expressions are never used (Pasero and 
Sabatier, 1997) (Pasero and Sabatier, 1994) (Mil- 
haud, 1994) (Godbert et al., 1993). 
3 Using ILLICO for a rehabilitation 
system 
What is the state of the art in the domain of 
AAC for autistic persons ? 
We have carried out a survey in the field of cur- 
rently available communication aids for autistic per- 
sons, to try to determine the qualities and short- 
comings of these systems. In fact, the interna- 
tional state of the art in this domain is rather 
poor. We can mention "Speaking Dynamically", 
"Boardmaker" (Mayer-Johnson, USA) as well as 
"Talk:About" (Don Johnston, USA). These systems 
use boards and picture communication symbols to 
compose picture sentences, some of them use a set 
of predefined sentences. We must also mention the 
"Facilitated Communication" method, which alms 
to help the user to use a keyboard of a computer to 
express himself by words and sentences. But Facili- 
tated Communication is devoted to persons who are 
physically unable to communicate, but do not have a 
difficulty with communication at the cognitive level. 
We must also note that ordinary computer assisted 
language learning systems can seldom be used by 
autistic persons for they require intuitive cognitive 
knowledge which is often lacking in autistic persons. 
Why is ILLICO relevant to the development 
of a language rehabilitation software ? 
First of all, we think that the two characteristics of 
ILLICO described in part 2 are big assets for the 
development of a language rehabilitation software: 
60 
I 
• In the guided mode, the user is led step by step 
during the construction of each sentence. This al- 
lows him to compose rapidly, with minimal cog- 
nitive load, sentences which are always correct at 
each level; this ensures also that the system never 
jams, i.e always "understand" what the user says. 
So the user doesn't become discouraged by fruitless 
attemps. The guided mode allows to begin rehabil- 
itation as soon as possible, even for very young or 
seriously disabled children. 
• The modularity of ILLICO is also a great asset, 
because it makes:it easy to define various exercises 
about language with different levels of difficulty: by 
using linguistic modules (lexicon and grammar) with 
broad or restricted coverage, by allowing or not the 
guided mode, we Obtain a lot of different exercises, 
among which one; can choose the one which is suit- 
able for each user's capacities. 
• Moreover, the ILLICO system, through the mod- 
ularity of its knowledge bases (defined at every level 
of well-formednesS), makes it possible to propose dif- 
ferent steps in the language acquisition or rehabili- 
tation process: 
- a lexical step which proposes exercises that allow 
acquisition of words and expressions of language in 
agreement with the subjects proposed; 
- a syntactical step which allows the acquisition of 
grammar rules by controlling the syntactic correct- 
ness of sentences; 
- a conceptual step which allows the user to acquire 
common sense rules by controlling the semantic cor- 
rectness of sentences; 
- a contextual step which makes it possible for the 
user to learn how to link sentences with the "real" 
world; at this step, the system verifies that the sen- 
tences make reference to objects actually present 
in the world; this arises in particular when using 
pronominal references or definite descriptions. 
From the survey we have made of the existing soft- 
ware in AAC, we are convinced that the sound tech- 
niques from artifiCial intelligence and computational 
linguistics provided by the ILLICO system are very 
interesting and innovative in the field of communi- 
cation aids for autistic persons: in the elaboration 
of our language rehabilitation software described be- 
low, our method has consisted in adapting existing 
technical solutions (graphic interfaces, etc.) to the 
needs of autistic users, by integrating ILLICO's tech- 
nology. We believe that the language systematic in- 
teractions (between lexicon, syntax and semantics) 
carried out by ILLICO are crucial for the treatment 
of language and cognitive disorders, i.e. can help 
users to improve their language and cognitive skills. 
In what follows, we first describe the functionality of 
our EREL system, then we detail one of the activ- 
ities proposed, and describe some specific elements 
of NLP required for its development. 
4 Functional description of EREL 
The El:tEL system provides a set of user-friendly ed- 
ucational play activities (logic games or scenarios), 
designed from communication and language training 
and learning exercises, and designed to stimulate, 
encourage, and help users to employ common lan- 
guage to build up an everyday language dialogue in 
interaction with the system, within a modular and 
multimedia context. Generally speaking, users will 
be able to express themselves on the subject pro- 
posed by the activity, with the assistance of the sys- 
tem (guided composition O f sentences) or freely (free 
composition). The software has been designed as a 
multi-level and multi-user system: a system flexi- 
ble enough to be adapted and to respond to specific 
needs according to the user's skills, which depend on 
his level of language and cognitive development, and 
his degree of autonomy. 
The set of activities proposed by EREL 
The activities to be chosen are dialogues on scenar- 
ios or on logic games: 
• In the first case, a scenario is illustrated by a pic- 
ture or a photograph; the user comments on with 
sentences describing what he sees on the screen. Pic- 
tures are a medium for language, and allow a child to 
enter a world by playing. The objective of this kind 
of activity is to incite the user to build simple sen- 
tences on a theme, and to develop the child's ability 
for naming, categorising or generalising an idea. 
• The second activity concerns dialogues on logic 
games based on pictures or puzzle pieces. Users play 
by means of sentences: on one hand, they may com- 
pose orders to achieve a goal (for example to move 
a piece of a puzzle), or they may comment on the 
progress of the game. The aim of this activity is the 
verbalisation of the action. The idea is to encourage 
the children to use language for doing tasks compos- 
ing a logical sequence of actions. 
These various activities are especially designed to 
help medical staffs in the evaluation and the reha- 
bilitation of the users' abilities to: 
61 
- Associate a word with a picture, generalise a 
concept, an idea, 
- Work on space locating and logical constructions, 
illustrated by pictures and by the movement of puz- 
zle pieces on the screen according to the actions ex- 
pressed by the users. 
All this work is carried through a common language 
dialogue. 
Various working modes 
The El:tEL system has a modular architecture, 
which allows to select the linguistic complexity of 
the activity, for a gradual work: each activity uses, 
among a set of available grammars and lexicons of 
gradual complexity, those from which the sentences 
Will be constructed. It is therefore possible to use 
the system for the education of children with dif- 
ferent levels of development, attention or reading 
skills. Besides, for each activity, the system pro- 
poses a set of functionalities responding to different 
requirements and competence levels, in accordance 
with the work that is expected to be done by the 
user. The working modes can be: 
- Discover linguistic components (the lexicon, the 
conceptual model, etc.), to enable users to famil- 
iarise themselves with words and concepts. 
- Produce text related to the scenario or game 
components. 
- Study the logical representation of the sentences' 
semantics: for some of the proposed activities, the 
system provides a graphic representation of the se- 
mantics of the resulting sentences (i.e. their under- 
lying meaning), using a simple and logical graphic 
formalism. In the opinion of the doctors we have 
consulted about this project, this access to a seman- 
tic representation of the sentences is extremely in- 
teresting with regard to the treatment of cognitive 
disorders. 
The operating modes and interfaces 
The input/output devices are defined to respond to 
the users' needs to the best, and to optimise their in- 
teractions with the system. In the guided mode, the 
software is operated by means of a series of graph- 
ics selections on the screen. The system can be used 
with or without an assistant, depending on the user's 
autonomy. Besides, the software is a real multime- 
dia system, each activity being illustrated through 
several coordinated methods: writing sentences on 
the screen and synthesising them orally, graphic rep- 
resentation on the screen. This multimedia nature 
of the software means that the different media are 
coordinated and organised in a coherent way, and 
this seems to be a crucial point for persons with lan- 
guage, cognitive and motor disorders. 
5 Example: an exercise proposed by 
EREL 
Presentation 
One of the exercises proposed by the software con- 
sists in putting and moving objects on a board. A 
child has a stock of objects that he can put on a 
checker board, permute, move or stow away. He 
gives orders to the system using natural language 
sentences and he can see immediately on the board 
the effects the sentences have. In addition to the lin- 
guistic learning, this type of exercise makes it also 
possible for the child to develop his capacity to lo- 
cate himself in the space. The interface looks like 
this: 
Son AcUvlte Nlvoou £onlrolntos 
Suites possibles : 
I $ 4 S &! © 
A 
II~sorlllle 
I ston I 
I Continued 
"E-change le cargo nolr avec le fond ... 
Figure 1: Example of a logic game in EREL 
Here, in the French version, the user has begun a 
sentence Echange le carrd noir avec le fond... (Per- 
mute the black square with the circle...) and the sys- 
tem, according to the contextual situation, proposes 
the possible words to be selected: blanc, gris, noir 
(white, grey, black). 
Here are some examples of sentences corresponding 
to different levels of difficulty: 
- Permute the grey circle with the black triangle. 
- Put the white circle in the square A5. 
- Put the white triangle under the pawn which is 
in the square B4. 
- Put the grey triangle at the left of the pawn which 
is situated under the pawn which is ... 
The different levels that we have defined correspond 
to levels of difficulty lying in the linguistic forms pro- 
posed (broader and broader coverage and finer and 
finer sentences decomposition in the guided mode) 
and, in parallel, to an extension of the cognitive pos- 
sibilities (in particular, in space locating). 
62 
Concerning linguistic forms, the system proposes, 
for example, graduated ways to designate objects: 
- At the simplest level, an object is necessarily des- 
ignated by its shape and its color (the black circle) 
and plurals are not allowed. 
- At higher levels of difficulty, the use of relative 
clauses is allowed and an object can be designated 
by its position (the circle which is in the square A3); 
plurals (the circles) and the generic word pawn are 
also allowed. 
- Finally, the Use of pronouns (in particular, clitic 
pronouns like in put them in the square A3) makes 
it possible to designate objects referentially. 
Concerning space locating, several levels are also 
possible to point out a square or a position. 
- A first level proposes to point out a square sim- 
ply by clicking on it; then, the system completes the 
sentence of the ttser with a noun phrase correspond- 
ing to the expression of the position designated that 
way. 
- At higher levels, a square can be identified by its 
position on the checker board or by its content (the 
square which contains the black round). At first, the 
positions are absolute (the square A4), then relative 
(at the left of the square A4, above ...). 
Concerning composition within the guided mode, we 
have defined two levels: a first one, where the whole 
syntagms are not decomposed and are considered 
as final expressions of the grammar, and a second 
one where the decomposition is made at the level of 
the words. In the first case, the child has to choose 
first of all a verb (put), then, if necessary, a whole 
noun phrase (the black triangle), and then, if neces- 
sary, another whole noun phrase (in the stock). This 
mode is only possible in the case where the linguistic 
coverage is reduced (no relative, for example) and it 
is very useful for the child to discover the abilities of 
the system. 
Some technical points 
The linguistic levels proposed in this exercise do not 
create any problem as for the linguistic surface forms 
which are very simple. 
A grammar has been developed which describes the 
highest level of difficulty. In order to compute lower 
levels of grammars, some rules of this grammar can 
be dynamically switched off according to the value 
of a global variable coding the level chosen by the 
user. A lexicon and a set of semantic composition 
rules have been developed and are used in the same 
way. 
The conceptual model describes the world of the ap- 
plication in terms of domains of objects and possi- 
ble relations between them. In order to take into 
account spatial expressions like above the square B4 
or at the left of the pawn which..., we have speci- 
fied which objects can be categorised as a place and 
which ones cannot. Following (LePesant, 1996), 
we think that not all the words which can appear 
in spatial expressions are places, but only some of 
them like square, stock. This allows us to distin- 
guish between the correct spatial expression in the 
stock and the incorrect (according to our applica- 
tion) one *in the triangle. In order to describe the 
conceptual features of correct expressions like above 
the triangle, we consider this sort of prepositions as 
functions which, applied to an object which is not 
a place, give a place as result: thus, the expression 
above the triangle can be used with a verb like put 
which requires a place as a complement (put the grey 
circle above the triangle). 
Some forms need a more elaborated semantic pro- 
cessing and, in particular, definite descriptions like 
the black circle, the pawn which is at the left of the 
square which contains the black triangle. Following 
(Russel, 1905) and (Strawson, 1950), we consider 
that a definite description refers to one and only one 
object in the context. In the type of applications de- 
scribed here, the object so designated must be iden- 
tified by the system in order to act on it and the 
consequences of the action upon the object must be 
taken into account in the representation of this ob- 
ject in the context. All these operations are made 
by the contextual module. Moreover, as we exposed 
in (Mouret and Rolbert, 1996), the treatment of 
definite descriptions when used in the guided mode, 
introduces particular constraints: because the sys- 
tem must propose correct sentences only (and be- 
ginning of correct sentences only), it has to know as 
early as possible (and actually in advance) which def- 
inite descriptions are correct according to a partic- 
ular context. These constraints require the system 
to apply specific processes to the context in order to 
know which objects can be designated by a definite 
description and which cannot: if there is no such ob- 
jects in the context, then no definite description can 
be produced. If such objects exist in the context, 
the system can produce definite descriptions which 
must agree with the description of these objects. 
Finally, the actions underlying the sentences have to 
be computed; their treatment introduces the general 
problem of the "semantics of actions". 
63 
6 Conclusion 
In this paper, we have presented EREL, a language 
education and rehabilitation system for autistic chil- 
dren, developed from the generic ILLICO system. 
This software fully uses the natural language pro- 
cessing techniques provided by ILLICO, and in par- 
ticular the principles of modularity and guided com- 
position. We have shown why guided composition is 
especially relevant to the development of communi- 
cation aids, and how the use of ILLICO makes it pos- 
sible to develop software which can help users to im- 
prove their language and cognitive skills. More par- 
ticularly, we have detailed through an example how 
the modularity of ILLICO allows us to define several 
language rehabilitation exercises which have differ- 
ent levels of difficulty from a linguistic and cognitive 
point of view. The development phase of EREL has 
shown that the particularities underlying this type 
of applications can easily be incorporated inside IL- 
LICO. 
The system is intented to be evaluated through the 
clinical and cognitive evolution of the children, first 
by medical personnel who will use standard evalu- 
ation methods, and also by the families, who will 
be able to daily test the appropriateness of the sys- 
tem for their children. Until now, it has not been 
possible to carry out a several months evaluation of 
the expected therapeutic effects on the population: 
presently, a medical team just begins to use a proto- 
type of the system with autistic-like and psychotic 
children. 
'7 Acknowledgments 
The ILLICO project has been partially funded by 
the French Ministate de la Recherche, and Conseil 
Rdgional Provence-Alpes-CSte d'Azur. The EREL 
project is partially funded by the Conseil Gdndral 
des Bouches-du-RhSne. 

References 
R. Garroux, M. Bonvarlet, and Chalsemartin. 1989. 
L'ordinateur l'dcole et l'hpital de jour. Revue 
l~ranaise de Psychiatrie, 2, February. 
E. Godbert, R. Pasero, and P. Sabatier. 1993. 
Natural Language Interfaces: Using Conceptual 
Constraints. in G. Salvendy and M.J. Smith 
(eds.). Fifth International Conference on Human- 
Computer Interaction (HCI'93) , Orlando, Else- 
vier, August. 
E. Godbert. 1996. Rddducation du langage et de la 
cognition chez des enfants handicapds : une appli- 
caion du traitement automatique du langage na- 
turel. Technical report LIM, Marseille, France. 
P. Howlin. 1989. Changing approaches to communi- 
cation training with autistic children Br. Disord. 
Commun., 24. 
D. Le Pesant. 1996. Un dictionnaire des noms com- 
muns de lieux. Technical report LLI and Univer- 
sitd d'Evry, France. 
R.G. Maurer, and A.R. Damasio 1982. Childhood 
autism from the point of view of behavioral neu- 
rology. Journal of autism and developmental dis- 
orders, 12, 2. 
G. Milhaud 1994. Un environnement pour la com- 
position de phrases assistde. Thse de doctorat, 
Laboratoire d'Informatique de Marseille, Univ. 
Aix-Marseille II. 
P. Mouret, and M. Rolbert 1996. Referring to the 
Context in a Guided Composition System Pro- 
ceedings of the Discourse Anaphora and Anaphor 
Resolution Colloquium (DAARC96), pages 507- 
518, Lancaster University, England. 
E.M. Ornitz 1974. The modulation of sensory input 
and motor output in Autistic children. Journal of 
autism and childhood schizophrenia, 4,3. 
R. Pasero, P. Sabatier 1994. ILLICO for Natural 
Language Interface Language Engineering Con- 
vention, (LEC). 
R. Pasero, and P. Sabatier 1997. Concurrent 
Processing for Sentences Analysis, Synthesis and 
Guided Composition. to appear in Lecture Notes 
in Computer Sciences, Springer. 
B. Russel. 1905. On denoting Mind, pages 479-493, 
14. 
P.F. Strawson. 1950. On referring, Mind, pages 
320-344, 59. 
