,C ..~ 
1965 International Conference on Computational Linguistics 
SOME CO~ONENTS OF A PROGRm',\[ FOR DYNAMIC MODELLING 
OF HISTORICAL CHANGE IN'LANGUAGE 
Sheldon Klein 
Carnegie Institute of Technology 
Pittsburgh, Pennsylvania 15213 
USA 
and 
System Development C.Orporation 
Santa Monica, California 
USA 
\. 
~o 0 ." ,0 <>..\ 
2F 
Klein i. 
SOME COMPONENTS OF A PROGRAM FOR DYNg.MIC MODELLING 
OF HISTORICAL CHANGE IN LANGUAGE 
Sheldon Klein 
ABSTRACT 
A system that is to serve as a vehicle for testing models 
of language change is being programmed in jOVIAL. Inherent in 
the design of the system is the requirement that each member of 
a speech community be represented by a generation grammar and a 
recognition grammar. The units of interaction in a simulation 
are conversations. Grammar rules may be borrowed or lost by 
individuals during the course of a simulation. The rules them- 
selves need not be limited to those suggested by a particular 
theory of lanquaqe; also, they may refer to any or all levels 
of linguistic phenomena. Extralinquistic factors pertinent 
to language change may be incorporated in simulations. 
Klein I. 
1.0 The Simulation System 
A general simulation system which is to serve as a device for 
testing of hypotheses about language change through time is being 
program~ned in JOVIAL, an ALGOL language, and is partially operation- 
1 a! on the Philco computer (4). The basic assumptions about the 
nature of language change inherent in the design of the Program 
include the notion of generation grammar, Bloomfield's concept of 
speech community (i), and Sapir's concept of genetic drift (5). 
Aside from these built in concepts, the program is designed as a 
vehicle for testing models of language change as a function of 
variables selected at the discretion of an experimenter. It is 
intended that the simulation system be sufficiently flexible to work 
with either transformational or stratificational models of language; 
to simulate the interaction of members of a speech community among 
themselves and with members of other communities; to model special 
relations among particular members, e.g. family groups and social 
classes; to simulate multilanguage acquisition; and to model the 
transmission of language from generation to generation. 
A basic assumption of the simulation system is that the 
interaction among members of a speech community is the prime 
1This research is supported in part by Grant ~4}{-07722, National 
Institutes of Health, United States Public Health Services (to 
the Carnegie Institute of Technology). 
Klein 2. 
focal point of language change. Each member of a speech community 
or sample from a speech community is represented by boZh a generation 
grammar and a recognition grammar. Members of a community who are 
familiar with more than one language may be represented by additional 
gray,mars. The contents of the grammars may vary among individuals. 
Grammars of newborn children would be empty. An adult entering 
a new community as a speaker of an alien language might acquire an 
empty recognition and generation grammar to supplement the nonempty 
ones representing the languages he knows. 
The basic units of interaction are speech forms produced in 
response to other speech forms. ~ primary function of the system 
is to simulate conversations among members of a speech community. 
During the course of a conversation, one individual will generate 
a form and another will attempt to parse it. Should the parser's 
rules be inadequate for the task, he mayborrow the necessary rules 
from the generation grammar of the speaker, and perhaps use it when 
it is his turn to speak. Note that a bilingual speaker might 
attempt parsings with rules from all of his grammars. 
Many decisions within the simulation system are made with 
the use of random numbers and functions governing the transition 
from one state of events to another. Monte Carlo techniques will 
be used in conducting simulations. Basically, the term refers to 
the use of random elements to solve essentially deterministic 
problems which may be too complicated to solve by deterministic 
methods. Accordingly, to evaluate the predictions of such a system, 
it is essential to determine the effects of different choices of 
Klein 3. 
random numbers numbers upon the results, if the model is deter- 
ministic, the results of repeated trials relying on different 
inputs of random numbers should be similar. 
2.0 Components 
The basic components of the the simulation system consist of 
a table containing the grammar rules and parameters associated with 
each individual in the simulation; a generation and parsing device 
that makes use of the grammars of interacting individuals; a table 
of functional relationships containing the rules of interaction 
pertinent to a particular simulation model; and, finally, a monitor 
program thai determines the flow of the simulation and the passage 
of time, and that periodically takes a census to inform the experiment- 
er of the changes occuring at various stages of the simulation. 
The first version of the simulation system is being constructed 
around the author's automatic essay paraphrasing system (2) which 
produces essaylike paraphrases of an input consisting of a restricted 
English text and an outline of the desired output essay. The 
syntactic style of the output is controlled by manipulation of 
parameters pertaininq to the frequency of usage of specific generation 
grammar rules (3). 
The table of functional relationships thai contains the definition 
of a particular model of language change might include rules express- 
ing such features as: 
i. Members of the same social group are more likely to speak 
to each other than.to members of other groups. 
2. Each time an individual interacts with a particular member 
of the community the probability of future interactions with thai 
Klein 4. 
member increases. 
More complex functions pertaining to particular socio-cultural 
conditions might also be used. 
Other functions might control the deletion of infrequently 
used grammar rules, or the shift of a grammar rule from a recognition 
qrammar to a qeneration qrammar. 
The monitoring system is designed to work with a mixed assort- 
ment of functional relationships pertaining to very different 
phenomena. At a given decision point the monitor scans the 
table of functions sequentially until it finds an applicable item. 
3.0 A Hand Simulation 
The nature and function of the basic components can be illustrated 
by a hand simulation of the flow of an extremely simple language 
model. 
Let the population contain six members: JOHN, ~4ARY, HELEN, 
PETER, HE~.~N and BABY. Let each have a separate generation and 
recognition grammar. Let each be assigned a status in the range 
of .Ol to .99, and let the letters A,B,C,D,E,F represent the grammar 
rules existinq in the community. (See table i.) The content of 
the rules is deliberately left unspecified. The rules may refer to 
semantics, syntax, morphology and/or phonology. Each rule is 
associated with a weighted frequency. A rule with a frequency weight 
less than a specified threshold value (.i in this simulation) can 
exist only in a recognition grammar. A rule with a frequency weight 
greater than or equal to the threshold must exist both in an individual's 
generation and recognition grammars. A rule existing in both grammars 
has the same frequency weight in each. A rule whose weight drops 
Klein 5. 
JOHN 
@ 
R 
T0,0 T0,1 T0,2 T0,3 TI,0 TI,I. T1, 2 
S .8 S .8 
A .5 A .47 
C .5 C .48 
D .5 D .53 
A .5 A .47 
B .04 B .02 
C .5 C .48 
D .5 D .53 
Y~RY 
G 
R 
S .7 S .72 S .7 S .64 
A ;5 
B .5 
D .5 
A .5 
B .5 
D .5 
E .O8 
Popular ion 
Table 1 
' Klein 6. 
HELEN 
G 
R 
T0,0 T0,1 TO, 2 TO, 3 T~, 0 TI, 1 TI, 2 
S .4 
B .5 
E .5 
B .S 
C .02 
E .5 
F .06 
S .4 
B .48 
E .5 
F .iS 
B .48 
E .5 
F .15 
PETER 
C 
R 
m 
S .3 
B .5 
E .5 
F .5 
B .5 
D .08 
E .5 
F .5 
S .32 S .38 
Table 1 Cont. 
S .38 
Klein 7. 
G 
R 
T0,0 
S .6 
B .5 
C .5 
B .5 
C .5 
D .02 
TO, 1 m ±0,2 
S .6 
B .53 
C .48 
B .53 
C .48 
A .07 
r~ =0,3 Ti,O Tl i 
S .6 
C .46 
B .57 
" 46 t., . 
A .05 
F .05 
TI,2 
BABY 
G 
R 
m 
S .4 S .4 
A .07 
B .07 
D .07 
S .4 
B .16 
A .05 
B .16 
D .05 
E .05 
F .05 
Table i Cont. 
• Klein 8. 
below a minimum value (.i in this simulation) is deleted from all 
qrammars. 
Table 1 contains a record of the various states of the speech 
community at time Ti,j, where i refers to a major cycle--a single 
individual's interaction with a variety of speakers, and where j 
refers to a minor cycle--the interval of an interaction with a single 
speaker. At each increment in the value i, the monitor randomly 
selects a member as speaker for a major cycle." The monitor then 
scans the population Sequentially to determine which members are 
to be auditors of the speaker. The determination follows the 
appropriate function contained in table 2. Each time an auditor 
is selected, the minor cycle time j is incremented by i. When 
the monitor has scanned the entire community, the speaker's turn 
is over and a new one is selected to ~ ~o~ the next major Cycle. 
At the beginning of each major cycle the j or minor cycle value 
is set to zero. The data in column T0, 0 of table 1 are startlng 
data supplied by the author. The data existing at Ti, j is used 
in comDutinq the state of events during Ti,j~ 1 . Blank entries 
in table 1 indicate that the state of events is unchanged from 
the previous interval. 
Table 2 contains the list of active rules refered to by the 
monitor during the course of the simulation. All computed values 
qreater than or equal to 1 are rounded to .99; values computed at 
less than or equal to 0 are rounded to .01~ in all cases, computed 
values are rounded to the second decimal place. 
Klein 9 
!. Probability of x speaking to y: 
Psi(X,y) = .I /Stalust.i(x) - staiusi.!(y) / 
2. Frequency weight of recognition rule m at time t after use in 
parsing: 
Ft(m) = Ft.l(m) (Ft_l(m) - relative frequency of m) in iparsinq at time t 
5 
3. Frequency of rule not used in parsing at time t: 
Fi(m) = Fi.l(m) - .02 
4. Threshold frequency weight for adding or removing a rule 
from a qeneration grammar: 
.i 
5. Threshold frequency weight for removing rule from a recognition 
gramma r : 
.01 
6. Status of speaker x after speaking to auditor y: 
Statust(x) = Statust.i(x) - (Statust.l(x) - otatus~_\](v) 5\[ 
Functions 
Table 2 
Klein i0. 
The simulation begins at time T0, 1 rather than at time T0, 0 for 
initialization purposes: 
T0,1 
The monitor selects }&~RY as speaker for the 0 cycle, and 
examines the list of potential auditors. The first candidate is 
JOHN. Accordinq to function 1 of table 2 the probability of }4ARY 
speakinq 9o /OHN is .i divided by the absolute value of the status 
difference of the pair: 
.i =-.99 (rounded) /.7 VS-/" 
Y~RY will speak to JOHN because the random n~mber qenerator of the 
monitor fails to yield a value greater than .99. Assume that ~v~Y 
generates the form: 
G(A, 2D) 
which is to be interpreted as indicatinq tha~ in the generation, 
JO~N is able to parse the rule A was used once, rule D twice, u.- 
form with his o~ ~ecoqnition~rules, and their frequency weights 
are a!tered accordinq to functions 2 and 3 in table 2. Rule A 
is computed as: 
.5 - (.5 - .33) : .47 
5 
Rule D as: 
.5- (.5 - .77) - .53 
5 
~ $OHN's recoqnition rules B and C were not used in the parsinq; after 
function 3 of table 2each of their weiqhts is decremented by.02. 
According to function 6 of table 2, }~RY's new status becomes: 
,7 - (.7- .8) _ .72 
5 
Klein ii. 
T0,2 
The monitor searches for Y~,RY's next auditor. ,,~,~°v~ is skipped 
as a,candidate. HELEN.is next. The probability of IVLARY speaking • 
io HELEN after function ! of table 2 is: ' ' 
.! : ! 
7.'7"2"- . 4:/- 
Assume HELEN is rejected as an auditor because monitor's random 
number generator produces a value greater than this. Assume that 
the next auditor candidate, PETER, is also rejected. The monitor 
then selects HERf~IAN as the next candidate. Now assume that HER\]v~N 
is selected as auditor after appropriate computations. Let f,\[ARY's 
generated utterance be: 
@(A, 2B) 
~.~,,,~ musfi borrow rule A froml YblRY's generation grammar to complete 
the parsing• Rule A enters HER~'~%N,s recognition grammar, by function 2 
of table 2, with a value: 
0- (0 - •33) : .07 
"" 5'"' 
Since this value is less than .i, it does not enter HERMAN's 
generation grammar. The new value of B is computed as: 
.5 - (.5 - .67) - .53 
5 
The rules not used in parsing are decremented by 02 HE~IIA~ s 
recognition rule D, accordingly, drops below the minimum retention 
value of •0~,and is deleted from his recognition grammar. 
Klein 12 
~,'~.RY's status is now computed as: 
.72 - (,:72 - .6) - 
5 
.7 
T0,3 
BABY is the next candidate for MARY's a'uditor. Assume that 
the monitor accepts BABY as a listener, and that ~RY tells him: 
G(A,B,D) 
BABY must borrow every pertinent rule from MARY's grammar, each 
with a frequency weight,computed by function 2 of table ?.,that is: 
0 - (0 - .33) - .07 
5 
~L%RY's new s~atus is now computed as: 
.7- (.7- .4) : .64 
5 
The monitor has exhausted the list of candidates for auditor and 
a new speaker must be selected randomly. 
'~71,0 
Let PETER be selected as the new speaker. Assume that IOHN 
and ~LiRY are rejected as auditors, but that HELEN is accepted: 
G(E, F) 
Rule E is inHELEN's recognition grammar and its new weight is: 
.5- (.5 - .5) - .5 
5 
remaining unchanged. The~weiqht:.of rule F is computed as: 
.06 - (.06 - .5) - .15 
5 
and after function 4 of table 2, £ enters her generation grammar. 
H=LsN's unused rules are decremented by .02 . 
PETER's new status is: 
.3 - .(,3 - .4) - .32 
5 
TI,I 
Klein 13. 
Assume HER},t%N is picked as PETER's next auditor, and PETER 
says : 
G(SB, F) 
Rule B is in HEF£4AN's grammar and its new frequency weight is: 
.53 - (.53 - .75) - .57 
,,,,, 
5 
Rule F is borrowed from PETER's grammar and enters HER~AN's generation 
grammar with a value: 
0 - (0 - .25) : .05 
5 
HER~N's unused rules are each decremented by .02 . PETER's new 
status is : 
.32 - \[.32 - .6) - .38 
5 
TI,2 
Assume the monitor determines BABY to be the next auditor, and 
that PETER generates: 
@(2B, E,F) 
Rule B is in BABY's. recognition grammar and it's new weight is: 
.O7- (,O7 - .5) -, ..~6 
5 "" 
Accordingly, rule B enters BABY's generation grammar. 
Rules E and F must be borrowed from PETER, and each enters 
Klein 14. 
BABY's recognition grammar with a weight: 
0 - (0 - .25) : .05 
'"5 ' 
The rules not used in the parsing are each decremented by .02 . 
PETER's new status is: 
.38 - (.38 - .4) : .38 
'5 
4.0 Discussion 
The preceding hand S'imulation should be sufficient to illustrate 
the operation of the simulation system. Anticipated computer 
simulations will involve 50 to 100 individuals, each associated with 
several<hundred grammar rules, iUnique ~ parsings can be obtained 
by using 6xistinq frequency weights to determine preferential 
applicability of 'rules. The functions contained in table 2 
can be qreatly extended in number and content. One miqht wish 
to add special rules for interaction between :parent ,'and Child, spouses, 
and among members of the same age'group, etc., plus a mechanism 
for determining the birth and death of various members. The status 
factor might be divided into weights refering to social status, aqe, 
geographical proximity and the like. 
The ideal test of the validity of a simulation is prediction. 
Hopefully, one miqht predict an attested state of a language from 
a model of an attested earlier stage. A major problem in such 
testing may be i i~.xlreme sensitivity of a model to the choice of 
parameter values and constants. For example, the constants in 
the functions of table 2 seem to have the effect of making BABY 
learn too quickly. One might use a higher rate of decay for unused 
Klein 15. 
rules to decrease the learning rate. The need for trial and 
error manipulation of values will increase with the complexity 
of a model. Accordingly, one might start with simple models, 
increasing the complexity by stages. 
The author's immediate research goal is to produce a stability 
simulation involving about 50 members,each associated with a 
simple phrase structure gra~nar of ~ ~nqmish, over a time span of 
3 or 4 qenerations--a simulation in which the language at the start 
of the simulation is reasonably similar to the language existing 
at the conclusion. 

References 

i. Bloomfield, L. Lanquaqe. New York: Holt, Rinehart, 19S3. 

2. Klein, S. Automatic Paraphrasing in Essay Format. 
Mechanical Translation. 
In press, 

3. Klein, S. Control of Style with a Generative Grammar. 
Language. 
In press, 

4. Klein, S. Dynamic Simulation of Historical Change in Language 
Using Monte Carlo Techniques. SP-1908, System Develo3ment 
Corporation, Santa Monica, December 1984. 

5. Sapir, E. Language. New York: Harcourt, Brace, 1921. 
