Non-directionality and Self-Assessment 
in an Example-based System 
Using Genetic Algorithms 
Yves Lepage 
Bahagian Sains Komputer, PP Sains Matc,natik & Sains Komputer, 
Universifi Sains Malaysia, I Igo0 Penang, Malaysia 
e-mail: lepage@cs.usm.jp 
re,arch done at ATR Interpreting Telecommunications Research L.abs, 
Hi -Gari-dai 2-2, Seika-cho, Soraku-gun, 619-02 Kyoto, Japan 
Abstract 
We show the application of an optimisation technique to 
natural language processing: genetic algorithms, thanks 
to the definition of a data structure called board and 
a formal distance. The system hms two interesting 
features: non-directionality, whict~ Ls more than hi- 
directionality, and self-assessment, independently of the 
inner knowledge. Results of experiments are presented 
and discussed. 
Topleal paper: Software for NLP 
Introduction 
The purpose of this article is to show that an engine 
b~ed on an optimisation teclmique, namely genetic al- 
gorlthras, can perform NLP t~usks: analysis and gener- 
ation in the frame of example-based approaches. But 
more than that, the system we have built has interesting 
properties: 
• it is truly non-directional, i.e. it performs more 
than bi-directional tasks; 
• it evaluates its results relatively to tile input, and 
not relatively to its internal knowledge. 
Two original facts make this possible: 
• the definition of a data structure, called board 
which is in essence bidirectional. It is the Kssocia- 
tiou of a sentence pattern m~d its linguistic struc- 
ture; 
• the definition of a ,distance on this data structure. 
1 Motivations 
1.1. Non-direetionality 
Our mMn motivation was to design a system where not 
only the formalism but also the system engine is bi- 
directional. In \[Lepage 91\], we sketched such an engine 
tbr a rule-based al)proach. 
From a theoretical point of view, such a system is 
also more interesting than a system where, although the 
formalism would be bi-directional, analysis and genera- 
Lion would be separate modules resulting from different 
compilations. 
In our sketch, a more general property than bi-direc- 
t:ionality mue,'go,¢: ,ro,>direclioTJagil!l. IGdirectionality 
is just the property of executing analysis and genera- 
tion with the same set of specifications, whereas nou- 
directionality allows more: a complete sentence and its 
complete analysis can be built from a partial specifica- 
tion of the sentence and a partial description of tile as- 
sociated structure. 
1.2 Self-assessment 
A second motivation lles in a flaw of rule-based systems 
using context-flee parsers, which is that they often fail 
to deliver a solution for trivia\[ reasons such ~ a word 
missing in a dictionary. On the contrary, our system al.- 
ways delivers an output for any input, would the solution 
be "bad". O\[" course, this woukl be of no meaning if the 
quality of outputs wotfid not be evahtated. IIence, when 
delivering a solution, the system scores it. 
Some machine translation systems viewed as expert 
systems may return an evaluation of their work in terms 
of their knowledge (grammar) \[Tong 89\], some other may 
evaluate the result according to thesaurus cl~ssificatlon 
and statistical frequencies \[Furuse and lida 92b1, but all 
these methods are specific. Ifere, on the contrary, the 
system delivers a score which is a formal distance be- 
tween the input and the output. Thus, it is independent 
of the linguistic representation chosen (dependency or 
constituency). '\[Fhis is not the case of a proposal such as 
\[llarrison el al. 91\]. 
'1?his score is a possible answer to the serious lack of 
assessment in natural language processing, ~m it, may ap- 
ply to any other system, leading to reliable comparisons 
of inl;rinsic performances 
2 Realisation 
Genetic algorith,ns constiL~lte a possible answer to tJie 
previous rnotivatior~s. 'l'hey are a collection of techniques 
for approaching the solution of optimisation problems 
\[Cloldberg 89\]. 
On tim contrary to usual programndng techniques 
which handle only one object, at. a time, genetic algo- 
rithms deal wit& a collection of individuals, called a pop- 
ulatiou. For each individual, one can compute a funct.ion, 
called the filT+ess flutction. Those individuals for which 
the fitness fm~ction is optimmn,'are the besl i.ndividuals. 
From two individm~.ls, one can derive two Imw imli- 
vldunls by cuttinv: lheu! itlt~ two pieces and gluing the 
616 
pieces back in the way ilhistrated in Figure 1. "J'}lis is 
crossover. Some random modiIication of the children 
may occur, accounting for mutalion to complete the ge- 
netic metaphor. 
N~AYl N 
nza, va 
\],'ignre 1: I~rinciple of crossover 
Tim previous operation can be rel)eated over a popu- 
lation a number of times so that populations follow one 
another. In the l,.~t generalion, the best individuals are 
hopefidly solutions of the optimisation problem at hand. 
In order to apply genetic algorithms to natural lan- 
guage processing, one hms to determine: 
• which data has to play the role of individuals; 
• consequently, what a population will be; 
• for an individual, wlmt its fitness is; 
• in a population, how individuals are selected for 
crossover and how it is performed; 
• how analysis and generation can be viewed as op- 
timisation problems on a population. 
'l?he meeting of our research interests with genetic ale 
gorithrns is a consequence of wirious awulable results. 
Firstly, the need for linguistic specification of corn- 
pnter grammars led to the proposal of an original data 
structure called board. This data structure is neutral 
with respect to ahalysis and generation. It will play the 
role of individuals. Boards rely on the data structure 
of forests, \[br which it is possible to define a crossover 
method. 
Secondly, tim stream of exainl)le-b~sed machine trans- 
lation is now well-established and justifies considering a 
collection of already parsed sentences as a population 
\[Sadler and Vendelmans 90\], \[Sate and Nagao 90\]. 
Finally, our recent work on distances, and especially 
on distances between uncompletely specified boards, led 
lls to the idea of looking in a data base for boards with 
the closest distance to a given board. This is an optimi- 
sation l)roblem and the tltness of any board will simply 
be a tilnction of its distance to the given board. 
In the following we will first describe the data struc- 
tures used. Then we will define the finlctions working 
on these data structures. 
2.1. Data struet;ures 
2.1.1 Individuals = Boards 
The board data structure \[Vauquois and Chaplmy 85\] 
w~s introduced ~s an answer to the problem of speci- 
fication of real-size graminars. A board is the associa. 
tion of a text with its corresponding, linguistic structure. 
Moreover, constraints express the linguistic validity of 
the fine-grained correspondences between different parts 
of the texts and of the structure \[Better and Zaharin 88\], 
\[Zaliarin and Lepage 92\]. As I a particular case, projec- 
t.iw~ constituency boards such as l,'igure 2 verify these 
cons{.raill ts, 
Boards would be of little nse if they would not allow 
the description <)f pattei'lls, llence, Figure 3 is also a 
.............. l ..... mod~l NP VP 
I .......... I .... 
pron AVP verb NP 
I ..... I .... 
~dv det NP nou\[i 
I 
Ii0un 
"Do you ~lreedy h~ve • registr~tion form?" 
Figure 2: A board 
valid board. It is similar to Figure 2, except that por- 
tions of the string and the tree have been replaced by 
variables (prefixed by a $ sign). These variables stand 
for tbrests, not only for trees -- tile point is important. 
Because it is always better to look for a unified view of 
S 
.............. I ..... 
raod~\]. NP 7P 
\[ ......... I .... 
pron Svb NP 
..... i .... 
de~ $i~oun 
"I)o you $vb ~ $noun" 
gigure 3: A board with variables 
objects, the string part and tim tree part are considered 
to be of the same data type, that of forest. As a matter 
of fact, a string is a forest with only o.lle level, and a tree 
is a forest with only one node on the highest level. Now, 
as forests are the underlying data type, variables stand 
naturally for subforests. On the string side, considering 
Variables as forests is by far more interesting than if they 
would instantiate with one word only. 
An interesting property about the board data struc- 
ture, and it is exactly why it has been devised, is that, 
because it is the association of a string (the text) and a 
(linguistic) tree, it is neutral with respect to the main 
natural language processing operations: 
i analysis (input: string, output: tree); 
• generation (input: tree, output: string). 
2.1.2 PoI)ulatlon = Data lmse of l)oards 
Our database of sentences is that of ATR telephone 
conversations. 'l?liese dialogues are telephone conversa- 
tions for a scenario where somebody calls at secretari. , to 
get, information about a coining conference he would like 
to atttend, l,'igure 4 is an excerpt from tliese dialogues. 
Ilello. 
- This is the Con/e,'enee OJJice. 
- Could ~jou tell me (:bruit the attendance fee for the 
Conference? If I at,ply for the (2on.ference now, how 
much is the atten&mce fee7 
- Yes. At pvese~lt the otte*tdailee Jee is 35,000 yell per 
person. IS yell appbj ne:ct month, it will be 40,000 yeF~. 
l,'igure ,1: An excerpt \['roill the A'\['IT, dialogues 
We kepl. I0 of these dialogues hi English. 'l'his rel;rc'- 
,<;e li I,S ') " " .,.it g('lltellCeS of ';\'hich 1.'30 are dill'erelll.. 
617 
The linguistic structures corresponding to the previ- 
ous sentences have been drawn by hand and scrupulously 
reviewed to ensure consistency. They are syntactic con- 
stituency trees and are exactly projective, which means 
that each leaf in the tree corresponds to a word in the 
sentence in the same order. 
As for illustration, all the trees and sentences in this 
paper are extracted from our data base of boards. Some 
representational choices have been made to limit the 
number of morpho-syntactic categories to 14 (and phrase 
types to 7) and to keep projectivity by all means. 
2.2 Pnnetions 
2.2.1 Fitness = Distance between forests 
We define the fitness of an element in a population (set 
of boards) ms the distance to a given input (a board) to 
the system. In other words, we have to define a distance 
between boards. A simple idea is to take the sum of the 
distances between the strings on the one band, and the 
trees on the other hand. As strings and trees are forests 
a distance on forests is required. 
The definition of a distance on forests is given below, 
with a, b being nodes, u, u ~, v, .v ~ being forests and . 
denoting concatenation of forests. 
dist(a(u').u, b(v').v) = m/n( 
di~t(a(u'), e) = di~t(~(,,'), b(,,')) 
disl(a,b) = 
dist(E,a) := 
dist(a,E) = 
di~t(a(,,'),b) + di~t(u,v), 
di~i(a(u'),e) "1- di~t(u,b(v').v), dist(Gb(u')) 
+ di~t(a(u').,,,u)) 
<u,t(<,, ~) 4- <u~(,,', ~-) 
dist(a,b) -I- dist(u',v') 
0 if a=b 
1 else (replaeeme)H) 
1 (insertion) 
1 (deletion) 
It is a direct generMisation of two classical distances 
on strings \[Wagner & Fischer 74\] and trees \[Selkow 77\]. 
Both distances answer the correction problem: what 
is the minimal number of typing operations needed to 
transform one object into the other one? In both dis- 
tances and their generalisation to forests, the typing op- 
erations are insertion, deletion and replacement 
An extension of the previous distance to forest pat- 
terns (i.e. forests containing variables) has been pre- 
sented in \[Lepage etal. 92\]. It is no longer a metric, so 
We call it a proximity score. With this score, the dis- 
tance between a variable and a constant object is zero 
by definition. Figure 5 gives an illustration (the unit is 
a one word or node difference). 
2.2.2 Crossover = Exchange of subfore.sts 
We turn now to crossover. The first question is how 
b,_..(Is a\]re selected in a population for crossover. 
It seems reasonable that those individuals with better 
fitness value should intervene more in the production of 
the next generation. Along this line, the simple following 
law gives the probability of a board i with fitness fl 
(some reciprocal of distance) to be selected for crossover: 
f~ Pi = 
As for crossover itself, it has to be defined on strings 
and on trees. 
On strings, be they chromosomes or sequences of bii,s, 
crossover is generally performed as ilhlstrated in Fig- 
ure 1. We could crossow'.r sentences following this sitnple 
l,riitclple (see Figure 6). 
be:fore crossover after crossover 
"Th~nk you vory much." "Thank you halp you." 
"May I help you?" iiMtty I very mtlch," 
Figure 6: Crossover on strings 
But we insist on keeping the unity of data struc- 
ture between strings and trees. So, we translate string 
crossover into forest terms: it is the exchange of the sis- 
ter forests of the crossover points. This can be applied 
directly to trees, see Figure 7. This technique is different 
from the exchange of subtrees ~s proposed in \[Koza 92\]. 
before crossover after crossover 
S S 
.... { ......... I ....... 
VP AVP VP AVP 
___{ .... { ....... { .... {._ 
*verb NP AVP ~dv verb VP AVP edv 
I| -J--I 
pron adv verb ~P adv 
i 
S pron 
........ I ..... 
mod~l *NP VP S 
1 --I ....... I ..... 
pron verb NP modal NP NP 
I I { 
pron proll pron 
Figure 7: Crossover on forests (crossover points are 
marked by *) 
Now, by keeping projectivity during crossover, only 
corresponding parts of strings and trees will be ex- 
changed. As a consequence, string crossover will allow 
exchange of |ruler substrings. To stun up, a board ob- 
tained by crossover will give a partially valid description 
of a possibly ungrammatical sentence (see Figure 8). 
before crossover 
7 .......... ........... 7 
{ .... { ..... \] 
I VP AVP I { ._.{ .... {__ { 
I *verb NP AVP adv J { { { { 
\] pron adv \[ 
l { 
l"Tbank you vory much." I 
........................ 
S 
........ I ..... modal ,~P 
VP 
{ --I-- 
proa verb NP { 
pron 
"M~y I help you." 
~ter crossover 
.... I ....... 
VP AVP 
..... { .... {._ 
verb VP AVP adv 
.A-. { 
verb NP ~dv { 
pron 
l"Thank help you very much." 
............................. 
S 
..... I ..... 
modul NP NP 
pron pron 
"H~y I you." 
Figure 8: Crossover on projective boards 
2.2.3 Optimisation problenl = Closest board in 
database 
The system built for the experiment implements a 
shnple genetic algoritlun. The starting polnllation is a 
set, of example boards, i, e. complete seidences with their 
crmIplel,e nssociated Ih~uisl,ic sl,','uctures, 
618 
disg( 
......... ....... i ...... ;, ....... 
I I I 
AVP I AVP 
AVP PP I AVP adv 
I --I- I I 
adv $1 NP \[ ~dv 
I I, 
qu ~.nt I 
I 
"llot $2" l +qtot; yet." 
............................... 
) = 
S S 
I I AVI' AVP 
...... I- --I-- disc:( 
ATP Pp AVP adv ) = 3 
I --I- I adv 
$ l NI' ~(Iv 
+ \] +=3 
qll&ll t 
,list;( "No~ a2" , "rIot yet;," ) = 0 
l,'igure 5: l)istance between two boards 
If an input board is given to Cite system, each board 
in the data base of examples carl be assigned a litness 
score: its distance to the input board. 
• When the input is a board where the Iingttistic tree 
is unknown (a variable), the output will be tim clos- 
est board containing the closest sentence with its 
,associated tree. This is a ldnd of analysis. 
........ I ..... I 
modal II|' VP \] 
I o-I-- l 
pron verb ~IP \[ 
1 E 
l,ron 1 I 
"may I help you" "H~y I help you?" I ................................................ 
input output; 
• When the input is a board where the string is un- 
known (a variable), the output will be the closest 
board containing the closest tree with its associated 
string. This is a kind of gencralion. 
............................................... 
S S 
........ I ............. I ...... modal NP VP modal NP VP 
pron verb NP pron verb NP 
I 1 
noun pron 
"$1" "M~y I help you?" 
............................................... 
input output 
• When the input is a board where both the sentence 
and the linguistic tree are partially specified (they 
contain variables), the otltl)ttC will be Che closest 
board containing a complete sentence and its con> 
plete associated linguistic structure. 
S S 
_--I ........... I ...... 
$1 }lP 7P mod,,.1 NP VP 
I I I .-I-- 
pron $2 pron verb NP 
1 
proll 
"13 help you $4" "flay I help you?" 
............................................... 
inpul; output 
We call tile last operatio\[l 7lon-dircclioTlal comple- 
lion. In fact, analysis and generation are only parCicuhu" 
cases of this operation. For in.stance, analysis is 11011- 
directional completion for a board will+ no w~riable iu 
the string part, and a w~riable as the tree parC. 
For each operation above, the external behaviour of 
tlt+.' system may be considered dilrerent, althotLt,:h the 
interttal behavio,jr is exactly tile san,,~. !n al',y ('a.'., L Cite 
..o'Utlmt is a board, built from pieces of the data base 
boards, and minimising the distance to the input, lC is 
important to stress the point that the ini)ut never enters 
Che data base of board:;. It is only used to compute Cite 
titness of each board in the data base in each generation. 
Figure 9 sunmlarises tile system and its functioning. 
l l- + 
Input 
• i 
, ,, , o i /\ i 
.... J L.,~_\ 
h----~l i 
I~l 
.... 
I ml/M I I Im\[~3 
• • 
,.inne~Rtlo,l I +.n{+ ,. tlo n~ ..... u.,2.r .tto,2j ) 
OLIIPII\[ 
Figure 9: A scheme of the system 
3 Experhrmntatior~ 
3.1 gXl)e.riments 
We tested the perl'ornlallce of the system ror analysis, 
generation and non-directional emnpleCio,. 
For analysis, ;~ board is extracted front the data base 
(call it reference board). A new board is built by as- 
sociating tile slming i)arL of the reference board with a 
variable as its I,ree pare. It, beconles the inImC \[.o the sys- 
Cent. Of course, the reference board is eliminated from 
\[,he d ataba.'m. 
A IirsC ineasure is given by +,he system itself: it: is rite 
fitness of the output, which is the distance between tile 
output and tile input. A second nleasure is tile distance 
I)eCwee.n I.he outlnll, and the reference board, which reo 
fleets the absolute qualiCy of the output. Moreov(!r, run- 
Limes have been measured. 
This procedure was carried out tbr eacll board of the 
¢la\[,a base so Chat average vahles (:oH\]d I)e ('oIlll)llCe+\]. 
There were 225 b,m'ds in the dal.a I>as,, 
619 
For generation, the same procedure was al)plied , but, 
of course, the tree part is kept in building tim input 
board. Pot non-dlrectional completion, an uneomplete 
board is automatically built by inserting variables at ran- 
dom positions in the string and tree parts of the reference 
board. 
3.2 R.esnlts 
Analysis Analysis gives an average error of about 9.2 
elements relatively to the exact output after thirty gen- 
erations. The average number of elements (nodes and 
words) in a board is 24.5, hence, the error rate is 38%, 
not a very good result. The fitness gives the average 
number of words wrong in the average string output by 
the system: around 3.2 words for a 8.5 word-long sen- 
tence. 
generation ~ity time"(in sec.) 3 ~. 6 ~--1--TKg~- T.g f 
-- 6 , ,1.48 I 13.09- 8,~" 
-'-"---i 2 ' 4.60'T~g- 13.26 
~---i ,q 1~ is,o= 
-t.13 ) 12.07 15,82 Tg .I ...... 
21 3.92 } 11.39 16.7-1 
24 I 3,68 \[ 10. 6--'---------~ 17.65 
Generation Generation is performed with better re- 
suits than analysis. The average error in the tree only is 
1.1 node for 16 node-heavy trees and the absolute error 
rate fails to 12%. However, as expected, generation is 
slower than analysb because more tree distance compu- 
tations are performed. 
3 4.64 \[ 9.49 8.16 
12 :t.56 ~5 - t s.;/~ i s ~ -------~753- 
18 ~ -- 18,99 
21 1---F3a-t,74 ~ i9.17 -4 
24 ~ 197a4-~ -----~-- --T7~.3~- a~ ".,o.9-i-- 
30 --T71a-F---VT~ 22.~T 
Non-direet:ional completion The following results 
must be considered ,as purely ilhtstrative, because the 
form of boards for non-directional completio i: un're- P !'d" 
stricted. As could be expected, because no p .... ~s com- 
plete in the input, quality is worse than for analysis and 
generation, although fitness appears to be quite good, 
I QUaIItV 
) in i).,i 
l G F--~o 7gY -~ 
15 1--~-A,-Y -~ 
u7 1------~ T--7:, ,---7 
7,40 
12,90 
15.'~2 
17.93 
lg,47 
18.95 
20.0l 
21.'17 
22,39 
3.,3 Discussion 
We will now discuss the advantages and drawbacks of 
our system. 
3.3.3. Non-dlre('tionallty 
The general fimction of the system is to build a corn- 
plete sentence and its complete associated syntactic tree 
from a partially specified sentence and a partially spec- 
ified tree. Hence, ana\[ysis and generation turn out to 
lie only l/articular cases of this general operation. This 
t'eatnre is what we called ares-directionality. It is more 
general than hi-directionality. Until now, we are not 
aware of any m~tural language processing system having 
this property. 
Frorn the applications point of view, non--directionali- 
ty allows one to envisage linguistically fotmded editing 
operations. For example, suppose we would like to re- 
place refltnd the fee by pay *lte fee back all over a text. 
We would like tile operation to apply for any tense of 
the verb. Tim Rlllowing I/oard could be used to retrieve 
all possible candidates. It says tllat we want a w'.rbal 
phrase (structural constraint) and that tile substring fl'.e 
must appear (string constraint). Of course, to perform 
such an operation, we would not advise the use of genetic 
algorithms . .. 
I verb ~1 I 
I I I refund $~ :fee I 
3.3.2 Assessment 
Because parts of the input ma~y be modified in tile 
ontput, assessnlent is necessary. The system delivers a 
score which is not directly connected to the knowledge of 
the system. 1I; is the distance between tile input and the 
output. Minimising this distance is precisely the ta.sk 
of the system. As this score is a theoretical metric be- 
tween structures, it is not'stud( to a particular represen- 
tation. It conld be applied to evahlate similar systems 
using difl'erent representations, for example dependency 
structures. 
3.3.3 Normalising efl'(m~ 
l)espite the previous points, important criticisms can 
still he addressed to the current system. 
l,;xperiulenLs carried out with int/ut sentences from 
outside tile data base have shown that the system has 
a normalising ell'ect: otfl;puts are cast to resemble sen- 
fences and trees fronl the database. This is a neg~l.b.e 
effect if a rreeq0ptlt syTstem is wanted. But,, it' a \]a%e 
enough data base is built and if standardisation is re- 
qulred, as is the ease with technical documents in many 
companies, this may be seen as a positive feature. 
:1.3.<I Colnimtai:ional limits 
A classical criticism of genetic algorithm ('oncerlls 
heavy computation time. llere, it is proportional to the 
tmnlber of examples in the data baso! This prevents us 
from using a big enough data base for any reasonable-size. 
application. 
As \['or spa(to., ollr iml>hmmntation of l.he sysl,enl re- 
lies on a ~.ooll)ox which makes extensiw.' use ol" dynamh: 
progralnl~6!!g (storing intermediate results to increase' 
speed). M...,Ioty !~izc li:lfits are rapidly reached. In ibis 
620 
implenlel,t,aLion, I, he space-time trade--off is a selmil;ive 
iSSLIO. 'lb relnedy 1)oth these, problems we are envisaging 
porting our progrmns on a parMlel machine. This does 
not add anything from the theoretical point of view, but 
genetic algorithms are obvious candidaLes for p;cra\[leli- 
sation. 
Conclusion 
'\])his p;~per h~L~; described t, he applical, ioI~ of an optimisa- 
Lion technique to naLural language processing tasks, i.e. 
analysis and generation. '\['he sysLe.m has been shown 
to haw~ two interesLing prope.rties: nonodirecLiona.lil.y, 
which is more than hi-directionality and self-assesslnent., 
independently of il;s internal knowledge. 
l~eferences 
\[Boitet and Zaharin 88\] Christian 13oi{x.¢ aml Zaharin 
Yusoff 
Representation trees and strhlg-tree corresl)}m- 
deAlces 
Proceedings of COLINU-8,~, pp 5,0-0d, \])u-. 
dapest., 1 {\]88. 
\[(7;oldberg 89\] D~tvid 1';. GoIdberg, 
Gene{ic AOorilhms iTt Search, Optimizalio~h 
and Machine Learning 
Addison Wesley Publishing Company, 1989. 
\[lIarrison el al. 91\] P. Itarrison, S. Abney, t';. P,b~ck, D. 
\[,'liekenger, C. Gdaniec, R. Grishman, 1). /lin- 
die, 1~. Ingria, M. Marcus, B. Santorini, T. 
Strzalkowski 
l;valuM;ing .qyntax Perf(n'mance of l~a.r.qer / 
Grammars of English 
Proceedings of the Workshop on lCvahtating 
Natural Language Processing Syslcms, ACL, 
1991, pp. ?-?. 
\[gin:use and lida 921>\] Furuse Osaulu and Iida \[lito~dfi 
An Example-.based Method for Transfer-driven 
Maehine Translation 
F'roceedings of the fourth lnternaHonal Confer- 
ence on Theoretical end Mcthodologieal lss~es 
in Machine Translation TMI-92, pp 139-150, 
Montrdal, 1992. 
\[i(oz~ 92\] John R. Koza 
Genelic Pvogrammin\[I - On lhe. Proyramvli~lg 
of Computers by Means of Nal~lral ,Veleeliol~ 
lVlI'£ Press, 1992. 
\[Lepage o," Vves Lepage 
• . :ring and Generating Contexl;.-gensiLive l,an- 
guages with Correst)ondenee ldentificat.ion 
G r arnmm's 
Proceedings of Ihe Pacific l?im .S'y~nposiu~u 
on Natural Lang'uage Processing, Singapore, 
November 1991, pp. 256-263. 
\[\[,epage et al. 92\] Yw~s Lepage, Furuse ()samu and li&~ 
Ilitoshi 
l{ela~;ion between ;t p;ttl:ern-matching operat.ion 
and it distance: ()n t;he pal,\[l 1;o reconcile two 
approaches in Nal;urM l,anguage }'rocee:;sing 
Proceedings of lke l,'irsl ,gingaporc In tern alioltat 
ConJ?.rencc o7~ lnlelliflenl Systems, .qil/gglpor,'. 
November 1992, Pl). 513~5"~. 
\[Sadler and Vendehnans90\] Victor ~adler and \]~onald 
VmMehnans 
Pilot imldelrlentation of a bilingual knowledge 
bank 
Proceedings of Colin./l-90, I\[elsinki, 1990, vol 3, 
pp..'1,t9-4U) 1. 
\[S;~Lo and Nagao 90\] Sat, o Sal, oshi and Nagao Mako{.o 
Toward bAemory-based Translation 
Proceedings of Colin\[I-90 , llelsinki, 1990, vol '2, 
pp. 247-252. 
\[Selkow 77\] S/;alfley M. Selkow 
The 'l'ree-to/lYee l;',diLing lh'oblem 
lnJbrmalion Processing Lellers, Vol. 6, No. {;, 
l)ecember 1977, pp. 184qg6. 
\[q'ong 89\] 'I'ol>g boone Chcong 
A daL~t-(h'iven coi|l;rol sl.r;t\[.egy for /'~l'~tlllll3glr 
writing systems 
Machilte Translation, 4(,1), December \[989, pp. 
177-197,. 
\[Vauquoi.s and Chal)puy 85\] l\]ernard Vauquois and .qyl- 
viane (3happuy 
.c-;t.at.ic lrl';tlltll\];tr.'-;: & forma.lism Ibr t, he descrip- 
tion of linguistic modeb; 
l'rocce.dings of the Co,.Jkren.ce on ThcoTrgical 
an d Melhodological Issues in Mackin e 7'ransla- 
lion, (2olgale Uni'versi@, llamilton, New York, 
Augttst ~.985, pp 298-322. 
\[Wagner & l,'ischer 74\] Robert A. Wagner and Michael 
d. Fischer 
The SLring-to-St;riug Correc@m Problem 
Jourual for lke AssociaHon of Compulin\[I Ma- 
chinery, Vol. 21, No. 1, Janu~try 1974, pp, 1(;~- 
173. 
\[Zaharin aml beI>age 92\] Zaharin Yusoff aml Yw'.s Lel)- 
age 
On I, he specification of ;xbs|,racL linguistoic strttc- 
tmes in formalisms for Machine Translation 
\['roceedings of lhc lnlernational b'ymposium on. 
Nat'ural Language Underslandin9 and A1, pp 
145-\[5.3, \[izuka, .luly 1992. 
621 
