Proceedings of the Human Language Technology Conference of the North American Chapter of the ACL, pages 328–334,
New York, June 2006. c©2006 Association for Computational Linguistics
Learning Morphological Disambiguation Rules for Turkish
Deniz Yuret
Dept. of Computer Engineering
Koc‚ University
 Istanbul, Turkey
dyuret@ku.edu.tr
Ferhan Tcurrency1ure
Dept. of Computer Engineering
Koc‚ University
 Istanbul, Turkey
fture@ku.edu.tr
Abstract
In this paper, we present a rule based
model for morphological disambiguation
of Turkish. The rules are generated by a
novel decision list learning algorithm us-
ing supervised training. Morphological
ambiguity (e.g. lives = live+s or life+s)
is a challenging problem for agglutinative
languages like Turkish where close to half
of the words in running text are morpho-
logically ambiguous. Furthermore, it is
possible for a word to take an unlimited
number of suf xes, therefore the number
of possible morphological tags is unlim-
ited. We attempted to cope with these
problems by training a separate model for
each of the 126 morphological features
recognized by the morphological analyzer.
The resulting decision lists independently
vote on each of the potential parses of a
word and the  nal parse is selected based
on our con dence on these votes. The
accuracy of our model (96%) is slightly
above the best previously reported results
which use statistical models. For compari-
son, when we train a single decision list on
full tags instead of using separate models
on each feature we get 91% accuracy.
1 Introduction
Morphological disambiguation is the task of select-
ing the correct morphological parse for a given word
in a given context. The possible parses of a word
are generated by a morphological analyzer. In Turk-
ish, close to half the words in running text are mor-
phologically ambiguous. Below is a typical word
 masal  with three possible parses.
masal+Noun+A3sg+Pnon+Acc(= the story)
masal+Noun+A3sg+P3sg+Nom(= his story)
masa+Noun+A3sg+Pnon+Nom DB+Adj+With
(= with tables)
Table 1: Three parses of the word  masal  
The  rst two parses start with the same root,
masal (= story, fable), but the interpretation of the
following + suf x is the Accusative marker in one
case, and third person possessive agreement in the
other. The third parse starts with a different root,
masa (= table) followed by a derivational suf x +l 
(= with) which turns the noun into an adjective. The
symbol  DB represents a derivational boundary and
splits the parse into chunks called in ectional groups
(IGs).1
We will use the term feature to refer to individual
morphological features like +Acc and +With; the
term IG to refer to groups of features split by deriva-
tional boundaries ( DB), and the term tag to refer to
the sequence of IGs following the root.
Morphological disambiguation is a useful  rst
step for higher level analysis of any language but it
is especially critical for agglutinative languages like
Turkish, Czech, Hungarian, and Finnish. These lan-
guages have a relatively free constituent order, and
1See (Oflazer et al., 1999) for a detailed description of the
morphological features used in this paper.
328
syntactic relations are partly determined by morpho-
logical features. Many applications including syn-
tactic parsing, word sense disambiguation, text to
speech synthesis and spelling correction depend on
accurate analyses of words.
An important qualitative difference between part
of speech tagging in English and morphological dis-
ambiguation in an agglutinative language like Turk-
ish is the number of possible tags that can be as-
signed to a word. Typical English tag sets include
less than a hundred tag types representing syntac-
tic and morphological information. The number of
potential morphological tags in Turkish is theoret-
ically unlimited. We have observed more than ten
thousand tag types in our training corpus of a mil-
lion words. The high number of possible tags poses
a data sparseness challenge for the typical machine
learning approach, somewhat akin to what we ob-
serve in word sense disambiguation.
One way out of this dilemma could be to ignore
the detailed morphological structure of the word and
focus on determining only the major and minor parts
of speech. However (Oflazer et al., 1999) observes
that the modi er words in Turkish can have depen-
dencies to any one of the in ectional groups of a
derived word. For example, in  mavi masal oda (=
the room with a blue table) the adjective  mavi (=
blue) modi es the noun root  masa (= table) even
though the  nal part of speech of  masal  is an ad-
jective. Therefore, the  nal part of speech and in-
 ection of a word do not carry suf cient information
for the identi cation of the syntactic dependencies
it is involved in. One needs the full morphological
analysis.
Our approach to the data sparseness problem is
to consider each morphological feature separately.
Even though the number of potential tags is un-
limited, the number of morphological features is
small: The Turkish morphological analyzer we use
(Oflazer, 1994) produces tags that consist of 126
unique features. For each unique feature f, we take
the subset of the training data in which one of the
parses for each instance contain f. We then split this
subset into positive and negative examples depend-
ing on whether the correct parse contains the feature
f. These examples are used to learn rules using the
Greedy Prepend Algorithm (GPA), a novel decision
list learner.
To predict the tag of an unknown word,  rst the
morphological analyzer is used to generate all its
possible parses. The decision lists are then used to
predict the presence or absence of each of the fea-
tures contained in the candidate parses. The results
are probabilistically combined taking into account
the accuracy of each decision list to select the best
parse. The resulting tagging accuracy is 96% on a
hand tagged test set.
A more direct approach would be to train a single
decision list using the full tags as the target classi -
cation. Given a word in context, such a decision list
assigns a complete morphological tag instead of pre-
dicting individual morphological features. As such,
it does not need the output of a morphological ana-
lyzer and should be considered a tagger rather than
a disambiguator. For comparison, such a decision
list was built, and its accuracy was determined to be
91% on the same test set.
The main reason we chose to work with decision
lists and the GPA algorithm is their robustness to ir-
relevant or redundant features. The input to the deci-
sion lists include the suf xes of all possible lengths
and character type information within a  ve word
window. Each instance ends up with 40 attributes on
average which are highly redundant and mostly irrel-
evant. GPA is able to sort out the relevant features
automatically and build a fairly accurate model. Our
experiments with Naive Bayes resulted in a signif-
icantly worse performance. Typical statistical ap-
proaches include the tags of the previous words as
inputs in the model. GPA was able to deliver good
performance without using the previous tags as in-
puts, because it was able to extract equivalent infor-
mation implicit in the surface attributes. Finally, un-
like most statistical approaches, the resulting models
of GPA are human readable and open to interpreta-
tion as Section 3.1 illustrates.
The next section will review related work. Sec-
tion 3 introduces decision lists and the GPA training
algorithm. Section 4 presents the experiments and
the results.
2 Related Work
There is a large body of work on morphological dis-
ambiguation and part of speech tagging using a va-
riety of rule-based and statistical approaches. In the
329
rule-based approach a large number of hand crafted
rules are used to select the correct morphological
parse or POS tag of a given word in a given context
(Karlsson et al., 1995; Oflazer and Tcurrency1ur, 1997). In
the statistical approach a hand tagged corpus is used
to train a probabilistic model which is then used to
select the best tags in unseen text (Church, 1988;
Hakkani-Tcurrency1ur et al., 2002). Examples of statisti-
cal and machine learning approaches that have been
used for tagging include transformation based learn-
ing (Brill, 1995), memory based learning (Daele-
mans et al., 1996), and maximum entropy models
(Ratnaparkhi, 1996). It is also possible to train sta-
tistical models using unlabeled data with the ex-
pectation maximization algorithm (Cutting et al.,
1992). Van Halteren (1999) gives a comprehensive
overview of syntactic word-class tagging.
Previous work on morphological disambiguation
of in ectional or agglutinative languages include
unsupervised learning for of Hebrew (Levinger
et al., 1995), maximum entropy modeling for Czech
(Haji c and Hladk·a, 1998), combination of statistical
and rule-based disambiguation methods for Basque
(Ezeiza et al., 1998), transformation based tagging
for Hungarian (Megyesi, 1999).
Early work on Turkish used a constraint-based ap-
proach with hand crafted rules (Oflazer and Kurucurrency1oz,
1994). A purely statistical morphological disam-
biguation model was recently introduced (Hakkani-
Tcurrency1ur et al., 2002). To counter the data sparseness
problem the morphological parses are split across
their derivational boundaries and certain indepen-
dence assumptions are made in the prediction of
each in ectional group.
A combination of three ideas makes our approach
unique in the  eld: (1) the use of decision lists and
a novel learning algorithm that combine the statis-
tical and rule based techniques, (2) the treatment of
each individual feature separately to address the data
sparseness problem, and (3) the lack of dependence
on previous tags and relying on surface attributes
alone.
3 Decision Lists
We introduce a new method for morphological dis-
ambiguation based on decision lists. A decision list
is an ordered list of rules where each rule consists
of a pattern and a classi cation (Rivest, 1987). In
our application the pattern speci es the surface at-
tributes of the words surrounding the target such as
suf xes and character types (e.g. upper vs. lower
case, use of punctuation, digits). The classi cation
indicates the presence or absence of a morphological
feature for the center word.
3.1 A Sample Decision List
We will explain the rules and their patterns using the
sample decision list in Table 2 trained to identify the
feature +Det (determiner).
Rule Class Pattern
1 1 W= c‚ok R1=+DA
2 1 L1= pek
3 0 W=+AzI
4 0 W= c‚ok
5 1  
Table 2: A  ve rule decision list for +Det
The value in the class column is 1 if word W
should have a +Det feature and 0 otherwise. The
pattern column describes the required attributes of
the words surrounding the target word for the rule
to match. The last (default) rule has no pattern,
matches every instance, and assigns them +Det.
This default rule captures the behavior of the ma-
jority of the training instances which had +Det in
their correct parse. Rule 4 indicates a common
exception: the frequently used word  c‚ok (mean-
ing very) should not be assigned +Det by default:
 c‚ok can be also used as an adjective, an adverb,
or a postposition. Rule 1 introduces an exception to
rule 4: if the right neighbor R1 ends with the suf x
+DA (the locative suf x) then  c‚ok should receive
+Det. The meanings of various symbols in the pat-
terns are described below.
When the decision list is applied to a window of
words, the rules are tried in the order from the most
speci c (rule 1) to the most general (rule 5). The  rst
rule that matches is used to predict the classi cation
of the center word. The last rule acts as a catch-all;
if none of the other rules have matched, this rule as-
signs the instance a default classi cation. For exam-
ple, the  ve rule decision list given above classi es
the middle word in  pek c‚ok alanda (matches rule
330
W target word A [ae]
L1, L2 left neighbors I [ iucurrency1u]
R1, R2 right neighbors D [dt]
== exact match B [bp]
= case insensitive match C [cc‚]
=+ is a suf x of K [kg g]
Table 3: Symbols used in the rule patterns. Capital
letters on the right represent character groups useful
in identifying phonetic variations of certain suf xes,
e.g. the locative suf x +DA can surface as +de, +da,
+te, or +ta depending on the root word ending.
1) and  pek c‚ok insan (matches rule 2) as +Det,
but  insan c‚ok daha (matches rule 4) as not +Det.
One way to interpret a decision list is as a se-
quence of if-then-else constructs familiar from pro-
gramming languages. Another way is to see the last
rule as the default classi cation, the previous rule as
specifying a set of exceptions to the default, the rule
before that as specifying exceptions to these excep-
tions and so on.
3.2 The Greedy Prepend Algorithm (GPA)
To learn a decision list from a given set of training
examples the general approach is to start with a de-
fault rule or an empty decision list and keep adding
the best rule to cover the unclassi ed or misclassi-
 ed examples. The new rules can be added to the
end of the list (Clark and Niblett, 1989), the front of
the list (Webb and Brkic, 1993), or other positions
(Newlands and Webb, 2004). Other design decisions
include the criteria used to select the  best rule and
how to search for it.
The Greedy Prepend Algorithm (GPA) is a variant
of the PREPEND algorithm (Webb and Brkic, 1993).
It starts with a default rule that matches all instances
and classi es them using the most common class in
the training data. Then it keeps prepending the rule
with the maximum gain to the front of the grow-
ing decision list until no further improvement can be
made. The algorithm can be described as follows:
GPA(data)
1 dlist ←NIL
2 default-class ←MOST-COMMON-CLASS(data)
3 rule ←[if TRUE then default-class]
4 while GAIN(rule, dlist, data) > 0
5 do dlist ←prepend(rule, dlist)
6 rule ←MAX-GAIN-RULE(dlist, data)
7 return dlist
The gain of a candidate rule in GPA is de ned
as the increase in the number of correctly classi ed
instances in the training set as a result of prepend-
ing the rule to the existing decision list. This is
in contrast with the original PREPEND algorithm
which uses the less direct Laplace preference func-
tion (Webb and Brkic, 1993; Clark and Boswell,
1991).
To  nd the next rule with the maximum gain, GPA
uses a heuristic search algorithm. Candidate rules
are generated by adding a single new attribute to the
pattern of each rule already in the decision list. The
candidate with the maximum gain is prepended to
the decision list and the process is repeated until no
more positive gain rules can be found. Note that if
the best possible rule has more than one extra at-
tribute compared to the existing rules in the decision
list, a suboptimal rule will be selected. The origi-
nal PREPEND uses an admissible search algorithm,
OPUS, which is guaranteed to  nd the best possible
candidate (Webb, 1995), but we found OPUS to be
too slow to be practical for a problem of this scale.
We picked GPA for the morphological disam-
biguation problem because we  nd it to be fast and
fairly robust to the existence of irrelevant or redun-
dant attributes. The average training instance has
40 attributes describing the suf xes of all possible
lengths and character type information in a  ve word
window. Most of this information is redundant or
irrelevant to the problem at hand. The number of
distinct attributes is on the order of the number of
distinct word-forms in the training set. Nevertheless
GPA is able to process a million training instances
for each of the 126 unique morphological features
and produce a model with state of the art accuracy
in about two hours on a regular desktop PC.2
2Pentium 4 CPU 2.40GHz
331
4 Experiments and Results
In this section we present the details of the data,
the training and testing procedures, the surface at-
tributes used, and the accuracy results.
4.1 Training Data
documents 2383
sentences 50673
tokens 948404
parses 1.76 per token
IGs 1.33 per parse
features 3.29 per IG
unique tokens 111467
unique tags 11084
unique IGs 2440
unique features 126
ambiguous tokens 399223 (42.1%)
Table 4: Statistics for the training data
Our training data consists of about 1 million
words of semi-automatically disambiguated Turkish
news text. For each one of the 126 unique morpho-
logical features, we used the subset of the training
data in which instances have the given feature in at
least one of their generated parses. We then split this
subset into positive and negative examples depend-
ing on whether the correct parse contains the given
feature. A decision list speci c to that feature is cre-
ated using GPA based on these examples.
Some relevant statistics for the training data are
given in Table 4.
4.2 Input Attributes
Once the training data is selected for a particular
morphological feature, each instance is represented
by surface attributes of  ve words centered around
the target word. We have tried larger window sizes
but no signi cant improvement was observed. The
attributes computed for each word in the window
consist of the following:
1. The exact word string (e.g. W==Ali’nin)
2. The lowercase version (e.g. W= ali’nin) Note:
all digits are replaced by 0’s at this stage.
3. All suf xes of the lowercase version (e.g.
W=+n, W=+In, W=+nIn, W=+’nIn, etc.) Note:
certain characters are replaced with capital let-
ters representing character groups mentioned in
Table 3. These groups help the algorithm rec-
ognize different forms of a suf x created by the
phonetic rules of Turkish: for example the loca-
tive suf x +DA can surface as +de, +da, +te, or
+ta depending on the ending of the root word.
4. Attributes indicating the types of characters at
various positions of the word (e.g. Ali’nin
would be described with W=UPPER-FIRST,
W=LOWER-MID, W=APOS-MID, W=LOWER-
LAST)
Each training instance is represented by 40 at-
tributes on average. The GPA procedure is responsi-
ble for picking the attributes that are relevant to the
decision. No dictionary information is required or
used, therefore the models are fairly robust to un-
known words. One potentially useful source of at-
tributes is the tags assigned to previous words which
we plan to experiment with in future work.
4.3 The Decision Lists
At the conclusion of the training, 126 decision lists
are produced of the form given in Table 2. The num-
ber of rules in each decision list range from 1 to
6145. The longer decision lists are typically for part
of speech features, e.g. distinguishing nouns from
adjectives, and contain rules speci c to lexical items.
The average number of rules is 266. To get an esti-
mate on the accuracy of each decision list, we split
the one million word data into training, validation,
and test portions using the ratio 4:1:1. The train-
ing set accuracy of the decision lists is consistently
above 98%. The test set accuracies of the 126 deci-
sion lists range from 80% to 100% with the average
at 95%. Table 5 gives the six worst features with test
set accuracy below 89%; these are the most dif cult
to disambiguate.
4.4 Correct Tag Selection
To evaluate the candidate tags, we need to combine
the results of the decision lists. We assume that the
presence or absence of each feature is an indepen-
dent event with a probability determined by the test
set accuracy of the corresponding decision list. For
example, if the +P3pl decision list predicts YES,
we assume that the +P3pl feature is present with
332
87.89% +Acquire To acquire (noun)
86.18% +PCIns Postposition subcat.
85.11% +Fut Future tense
84.08% +P3pl 3. plural possessive
80.79% +Neces Must
79.81% +Become To become (noun)
Table 5: The six features with the worst test set ac-
curacy.
probability 0.8408 (See Table 5). If the +Fut deci-
sion list predicts NO, we assume the +Futfeature is
present with probability 1−0.8511 = 0.1489. To
avoid zero probabilities we cap the test set accura-
cies at 99%.
Each candidate tag indicates the presence of cer-
tain features and the absence of others. The prob-
ability of the tag being correct under our indepen-
dence assumption is the product of the probabilities
for the presence and absence of each of the 126 fea-
tures as determined by our decision lists. For ef -
ciency, one can neglect the features that are absent
from all the candidate tags because their contribu-
tion will not effect the comparison.
4.5 Results
The  nal evaluation of the model was performed on
a test data set of 958 instances. The possible parses
for each instance were generated by the morpholog-
ical analyzer and the correct one was picked manu-
ally. 40% of the instances were ambiguous, which
on the average had 3.9 parses. The disambiguation
accuracy of our model was 95.82%. The 95% con -
dence interval for the accuracy is [0.9457, 0.9708].
An analysis of the mistakes in the test data show
that at least some of them are due to incorrect tags
in our training data. The training data was semi-
automatically generated and thus contained some er-
rors. Based on hand evaluation of the differences be-
tween the training data tags and the GPA generated
tags, we estimate the accuracy of the training data to
be below 95%. We ran two further experiments to
see if we could improve on the initial results.
In our  rst experiment we used our original model
to re-tag the training data. The re-tagged training
data was used to construct a new model. The result-
ing accuracy on the test set increased to 96.03%, not
a statistically signi cant improvement.
In our second experiment we used only unam-
biguous instances for training. Decision list training
requires negative examples, so we selected random
unambiguous instances for positive and negative ex-
amples for each feature. The accuracy of the result-
ing model on the test set was 82.57%. The problem
with selecting unambiguous instances is that certain
common disambiguation decisions are never repre-
sented during training. More careful selection of
negative examples and a sophisticated bootstrapping
mechanism may still make this approach workable.
Finally, we decided to see if our decision lists
could be used for tagging rather than disambigua-
tion, i.e. given a word in a context decide on the full
tag without the help of a morphological analyzer.
Even though the number of possible tags is unlim-
ited, the most frequent 1000 tags cover about 99%
of the instances. A single decision list trained with
the full tags was able to achieve 91.23% accuracy
using 10000 rules. This is a promising result and
will be explored further in future work.
5 Contributions
We have presented an automated approach to learn
morphological disambiguation rules for Turkish us-
ing a novel decision list induction algorithm, GPA.
The only input to the rules are the surface attributes
of a  ve word window. The approach can be gener-
alized to other agglutinative languages which share
the common challenge of a large number of poten-
tial tags. Our approach for resolving the data sparse-
ness problem caused by the large number of tags is
to generate a separate model for each morphologi-
cal feature. The predictions for individual features
are probabilistically combined based on the accu-
racy of each model to select the best tag. We were
able to achieve an accuracy around 96% using this
approach.
Acknowledgments
We would like to thank Kemal O azer of Sabanc 
University for providing us with the Turkish mor-
phological analyzer, training and testing data for dis-
ambiguation, and valuable feedback.
333
References
Brill, E. (1995). Transformation-based error-driven
learning and natural language processing: A case
study in part-of-speech tagging. Computational Lin-
guistics, 21(4):543 565.
Church, K. W. (1988). A stochastic parts program and
noun phrase parser for unrestricted text. In Proceed-
ings of the Second Conference on Applied Natural
Language Processing, pages 136 143.
Clark, P. and Boswell, R. (1991). Rule induction with
CN2: Some recent improvements. In Kodratoff,
Y., editor, Machine Learning  Proceedings of the
Fifth European Conference (EWSL-91), pages 151 
163, Berlin. Springer-Verlag.
Clark, P. and Niblett, T. (1989). The CN2 induction al-
gorithm. Machine Learning, 3:261 283.
Cutting, D., Kupiec, J., Pedersen, J., and Sibun, P. (1992).
A practical part-of-speech tagger. In Proceedings of
the 3rd Conference on Applied Language Processing,
pages 133 140.
Daelemans, W. et al. (1996). MBT: A memory-based
part of speech tagger-generator. In Ejerhead, E. and
Dagan, I., editors, Proceedings of the Fourth Workshop
on Very Large Corpora, pages 14 27.
Ezeiza, N. et al. (1998). Combining stochastic and rule-
based methods for disambiguation in agglutinative lan-
guages. In Proceedings of the 36th Annual Meeting of
the Association for Computational Linguistics (COL-
ING/ACL98), pages 379 384.
Haji c, J. and Hladk·a, B. (1998). Tagging in ective lan-
guages: Prediction of morphological categories for a
rich, structured tagset. In Proceedings of the 36th
Annual Meeting of the Association for Computational
Linguistics (COLING/ACL98), pages 483 490, Mon-
treal, Canada.
Hakkani-Tcurrency1ur, D. Z., Oflazer, K., and Tcurrency1ur, G. (2002).
Statistical morphological disambiguation for aggluti-
native languages. Computers and the Humanities,
36:381 410.
Karlsson, F., Voutialinen, A., Heikkilcurrency1a, J., and Anttila, A.
(1995). Constraint Grammar - A Language Indepen-
dent System for Parsing Unrestricted Text. Mouton de
Gruyter.
Levinger, M., Ornan, U., and Itai, A. (1995). Learning
morpho-lexical probabilities from an untagged corpus
with an application to hebrew. Computational Lin-
guistics, 21(3):383 404.
Megyesi, B. (1999). Improving brill’s pos tagger for an
agglutinative language. In Pascale, F. and Joe, Z., ed-
itors, Proceedings of the Joing SIGDAT Conference
on Empirical Methods in Natural Language and Very
Large Corpora, pages 275 284, College Park, Mary-
land, USA.
Newlands, D. and Webb, G. I. (2004). Alternative strate-
gies for decision list construction. In Proceedings of
the Fourth Data Mining Conference (DM IV 03), pages
265 273.
Oflazer, K. (1994). Two-level description of turkish
morphology. Literary and Linguistic Computing,
9(2):137 148.
Oflazer, K., Hakkani-Tcurrency1ur, D. Z., and Tcurrency1ur, G. (1999).
Design for a turkish treebank. In Proceedings of
the Workshop on Linguistically Interpreted Corpora,
EACL 99, Bergen, Norway.
Oflazer, K. and Kurucurrency1oz,  I. (1994). Tagging and morpho-
logical disambiguation of turkish text. In Proceedings
of the 4th Applied Natural Language Processing Con-
ference, pages 144 149. ACL.
Oflazer, K. and Tcurrency1ur, G. (1997). Morphological disam-
biguation by voting constraints. In Proceedings of the
35th Annual Meeting of the Association for Computa-
tional Linguistics (ACL97, EACL97), Madrid, Spain.
Ratnaparkhi, A. (1996). A maximum entropy model for
part-of-speech tagging. In Proceedings of the Confer-
ence on Empirical Methods in Natural Language Pro-
cessing.
Rivest, R. L. (1987). Learning decision lists. Machine
Learning, 2:229 246.
van Halteren, H., editor (1999). Syntactic Wordclass Tag-
ging. Text, Speech and Language Technology. Kluwer
Academic Publishers.
Webb, G. I. (1995). Opus: An ef cient admissible algo-
rithm for unordered search. JAIR, 3:431 465.
Webb, G. I. and Brkic, N. (1993). Learning decision lists
by prepending inferred rules. In Proceedings of the AI
93 Workshop on Machine Learning and Hybrid Sys-
tems, pages 6 10, Melbourne.
334
