Combination of an Automatic and an Interactive 
Disambiguation Method 
Masaya Yamaguchi, Takeyuki Kojima, 
Nobuo Inui, Yoshiyuki Kotani and Hirohiko Nisimura 
Department of Computer Science, Tokyo University of Agriculture and Technology, 
Nisimura, Kotani unit, 2-24-16 Naka-cho, Koganei, Tokyo, Japan 
Abstract 
In natural language processing, many methods have 
been proposed to solve the ambiguity problems. In 
this paper, we propose a technique to combine a 
method of interactive disambiguation and automatic 
one for alnbiguous words. The characteristic of our 
method is that the accuracy of the interactive dis- 
ambiguation is considered. The method solves the 
two following problems when combining those dis- 
ambiguation lnethods: (1) when should the inter- 
active disambiguation be executed? (2) which am- 
biguous word should be disambiguated when more 
than one ambiguous words exist in a sentence? Our 
method defines the condition of executing the inter- 
action with users and the order of disambiguation 
based on the strategy where the accuracy of the re- 
sult. is maximized, considering the accuracy of the 
interactive disambiguation and automatic one. Us- 
ing this lnethod, user interaction can be controlled 
while holding the accuracy of results. 
1 Introduction 
In natural language processing, many methods 
have been proposed to solve the ambiguity prob- 
lems(Nagao and Maruyama, 1992). One of those 
technique uses interactions with users, because it is 
difficult to make all the knowledge for disambigua- 
tion beforehand. That technique is classified into 
two types according to the condition of executing 
user interaction. One type(TypeA) is that the dis- 
ambiguation system executes interactions(Blanchon 
et al., 1995), (Maruyama and Watanabe, 1990), 
(Yalnaguchi et al., 1995). Another type(TypeB) is 
that users executes interactions(D.Brawn and Niren- 
burg, 1990), (Muraki et al., 1994). In thispaper, Ty- 
peA will be adopted because TypeB gives users more 
trouble than TypeA does. For example, in TypeB, 
a user may have to find where is wrongly analyzed 
in input sentences. 
In TypeA, the two following conditions must be 
determined: (1) when should interactive disam- 
biguation be executed? (2) which ambiguous words 
should be disambiguated when more than one aln- 
biguous word exist in a sentence? Considering the 
accuracy of tile analyzed result, they should be de- 
cided by both the accuracy of the interactive dis- 
ambiguation and that of tile autolnatic disambigua- 
tion. The traditional lnethods did not considered 
the accuracy of the interactive disambiguatiom For 
instance, the accuracy of the analyzed result may 
decrease in spite of executing the user interaction 
if the accuracy of the interactive disaml)iguation is 
low. 
In this paper, we propose the method to com- 
bine the interactive disambiguation and the auto- 
matic one, considering each accuracy. The method 
allows the disambiguation system to maximize the 
accuracy of the analyzed result. This paper focuses 
on the anabiguity caused by ambiguous words that 
have more than one mealfing. Section 2 represents 
preconditions for disamlfiguation. In Section 3, we 
descrihe the condition of executing the interactive 
disambiguation. Section 4 shows the procedure that 
decides the order of disamhiguation. The perfor- 
mance of the lnethod is discussed by the result of 
the sinmlation under assumhlg the both accuracy 
of the interactive disambiguation and the autolnatic 
one. 
2 Preconditions for Disambiguation 
This section describes preconditions for disambigua- 
tion and methods of the disamlfiguation. 
In this paper, the disambiguation for ambiguous 
words means that all ambiguous ones in an input 
sentence a.re disambiguated. Describing it. formally, 
the disambiguation is to decide one element of the 
following MS. 
MS = M1 x M2 x ... x slit, 
where an input sentence contains ! ambiguous 
words. Mi means the set of lneanings in the am- 
biguous word wi. 
Each disambiguation method has preconditions as 
follows: 
Interactive Disambiguation 
• In the interaction, the system shows explana- 
tions for each meaning of an ambiguous word to 
a user, who selects one explanation from them. 
1423 
• The system can calculate the probability where 
a user selects the right explanation. 
Automatic Disambiguation 
• The occurrence probabilities for each candidate 
can be calculated for preference. 
• The result is the candidate with the maximum 
occurrence probability. 
To show the iuformation mentioned above, candi- 
dates are expressed by the tree in Figure 1. This tree 
is an example in the case that an input sentence is "I 
saw a star.", which contains two ambiguous words 
'see' and 'star' and each word has two meanings. 
root 
771~1 1 77112 1711n 
Pdl, Pl Pd-~, P.~ Pdn, P, 
Figure 2: An example of the tree of candidates for 
one ambiguous word in an input sentence 
The accuracy of the interactive disambiguatiou 
/~ntr and that of the automatic disambiguation Pauto 
are defined as follows: 
root. 
see_l see_2 
Pdll Pdl2 
sta~_l stax_2 star_l star_2 
Pd2_l, Pll Pd22, P12 Pd21, P21 Pd22, P'_'2 
Figure 1: All example of the tree of candidates 
The depth of the tree expresses the order of dis- 
anfl)iguation. In Figure 1, the auabiguities are re- 
solved in the order from 'see' to 'star'. The occur- 
fence probability is calculated at each leaf node by 
the automatic disambiguation method. For exam- 
pie, PH expresses the probability for the candidate 
{see_l,star_l}. Furthermore, the accuracy of in- 
teraction is also calculated at the leaf node by the 
interactive disalnbiguation method. Pd~.l is the prob- 
ability where the meanillg of 'star' is 'staLl' and 
tim system shows explanations of 'star_l', 'star_2' 
for 'star' to a user a.nd (s)he selects the explanation 
of 'star_2'. At. Nodes besides leaf ones, only the 
accuracy of interaction is calculated. 
3 The Condition of Executing the 
Interactive Disambiguation 
3.1 Basic Idea 
At each node besides leaf ones, the disambigua- 
tion system decides which disambiguation method 
is used. Basically, the interactive disambiguation is 
executed when its accuracy is higher than the ac- 
curacy of the automatic disambiguation. First of 
all, let us consider the case where an input sentence 
contains one ambiguous word that has ~, meanings. 
Figure 2 shows the tree of candidates for this case. 
Pintr ---- £ PdiPi 
i 
P~uto = maxp~ 
The interactive disambiguation is executed, when 
the following condition is satisfied. 
Pintr > Pauto 
Considering tile condition more carefully, the ac- 
curacy of tile interactive disambigualion is iuflu- 
enced by the explanations that are showu t.o users. 
Thus tim accuracy may be improved by limiting to 
show some explanations to users. For example, this 
may be caused when the accuracy of roll is very low 
and a user may select mll wrongly by the higher 
similarity of the explanation for 11111 to other expla- 
nations. The autonmtic disambiguation corresponds 
to showing only one explanation to users in the in- 
teractive disanabiguation. Therefore the condition 
of executing the interactive disambiguatiou can be 
defined as the exceptional case of the limitation. 
3.2 The Accuracy at a Node 
In the case that the number of alnbiguous words is 
one as Figure 2, the accuracy of the deeper nodes be- 
low the root node needs not to be decided because 
they are leaf nodes. When more than two ambiguous 
words exist in an input sentence, a node may often 
have one that is not a leaf one. To calculate the ac- 
curacy of such a node, it is necessary to determine 
what kind of disambiguation will be executed at the 
deeper nodes. For instance, the disambiguation sys- 
tem has to fix each accuracy of node 'see_l' and 
'see_2' in Figure 1 to calculate the accuracy of the 
root node. Therefore, the definition of the accuracy 
at any node i is the recursive one. The accuracy of 
the interactive disambiguation Pintr(i) and that of 
the automatic disambiguation P~,to(i) at node i is 
defined as follows: 
1424 
Ptntr(i) = ~ pd(,nlM ) x P,(m) (1) 
rnEM . 
Pluto(i) = max(Pr(m)) (2) m E M 
where M is the set of the node directly under node 
i, pd(m\[M) is the accuracy of the interactive disam- 
biguation at node m, that is, the probability that a 
user selects m provided that the system shows ex- 
planations for all the elements of M to him(her). 
Pr(m) is the accuracy at node m and the definition 
is as follows: 
P~(,,) = 
(if the interactive disambiguation is 
executed a,t, node m) 
Pluto(,7/.) 
(if the automatic disambiguation is ex- 
ecuted at node m) 
Poccur(m) (if m is a leaf node) 
where/)occur(m) is tile occurrence probability of 
the candidate that includes nodes between the root 
node alld Ilode 7/l. 
When tile following condition is satisfied, the ill- 
teractive disanlbiguation is executed at node i. 
Pintr(i) > Pauto(i) (3) 
3.3 The Limitation of Explanations 
Ill user interaction, tile presentation of many expla- 
nations gives users trouble t.o select, one explanation. 
So it. is desirable that tile disambiguation system 
shows fewer exl)lanation to users, if possible. In this 
section, we describe the condition where the number 
of explanations is limited without losing the accu- 
racy of the analyzed result. 
By formula (1), the accuracy of the interactive 
disanlbiguation Piaster in the case of limiting the set 
of explanations AI ~ is defined as follows: 
max Z pd(m\[M -- M')P,(m) M ~ 
mEM-M ~ 
Pitntr(i) - if \]M - M'\[ > 1 
Pr(t) if IM - M'I = 1 
If fornmla (4) is satisfied, the set. of tile explana- 
tion M' is not shown to users in the interaction at 
node i. 
/~ntr(i) ~ Pi~ntr(i) (4) 
Furtherlnore, if Ill,l- M' I = 1, then tile automatic 
disambiguation is executed at. node i. Therefore, 
formula (4) implies fornmla (3). 
4 Determination of the Order of 
Disambiguation 
4.1 Procedure 
up to here, we have discussed ~l:amt r and Pluto under 
a certain order of disambiguation. Ill this section, 
we describe a procedure to decide the order of dis- 
ambiguation where the accuracy is maximum. 
The accuracy of the analyzed result may be differ- 
ent in each order of disambiguation, This is the rea- 
son that the disambiguation of one ambiguous word 
leads to constrain the meaning of other ambiguous 
ones. Therefore, the contents of the interaction may 
differ from each order of disambiguation. The or- 
der with the maximum accuracy is obtained in the 
following procedure: 
1. Calculating each occurrence probal)ility of can- 
didate for tile analyzed result by the automatic 
disambiguation method. 
2. Obtaining the accuracy in each order of (lisam- 
biguation based on the method described in the 
previous sections. 
3. Disanlbiguating by the order with the maximum 
accuracy. 
4.2 Example 
Ill this section, we illustrate the determination of ex- 
ecuting the interactive disambiguatioll and the order 
of disanlbiguation. The values at leaf nodes are the 
occurrence probabilities. Tile accuracy of the inter- 
active disalnbiguation is 0.9 at the any nodes. Since 
the number of ambiguous words is two, the num- 
ber of the order of disambiguation is 2! as shown in 
Figure 3, 4. 
root 
see_l see_2 
star_l star_2 star_l star_2 
0.10 0.10 0.05 0.75 
Figure 3: An example of tile order of disambigua- 
tion(1) 
To begin with, we intend to calculate what kind 
of disambiguation is executed at node 'star_l' 
and 'star_2', ill Figure 3. By fornmla (1), (2), 
~nt,.(see-1) and Pluto(see-I) are as follows (since 
both ambiguous words have two meanings, P\[ntr(i) 
= Pluto(i)): 
1425 
root, 
star_l star_2 
see_l see_2 see_l see_2 
0.10 0.05 0.10 0.75 
Figure 4: An example of the order of disambigua- 
tion(2) 
Pi,,t,.(see_l) -'- 0.9 x (0.75 + 0.05) 
= 0.72 
Pauto(see-1) -- max(0.75,0.05) 
= 0.75 
Because of Pi.~,.(see_l) < Pauto(See-1), the au- 
tomatic disambiguation is executed at. node see_l. 
Oil the other hand, at node see_2, P,,,t,.(see_2) and 
Pa.to(see-2) are as follows: 
Pi,~,.(see_2) = 0.18 
t~,to(see_2) = 0.10 
Pi,,tr(see_2) > Pa,,to(see-2) is satisfied. So the 
system interacts with users at this node. 
By the result of the above, Pi,t,.(root) and 
Pa,to(root) are as follows: 
Pi,,t,.(root) = 0.9(Pr(see_l) -t- P,.(see_2)) 
: 0.9(Pauto(see-1) + Pi,,t,.(see-2)) 
= 0.0(0.75+ 0.18) = 0.837 
Pauto(root) = max(Pr(see_l),Pr(see_2)) 
= max(0.75,0.18) = 0.75 
Therefore, the interactive disambiguation is ex- 
ecuted at. the root. node because of Pint,.( root ) > 
P~to( rOot ), and P~(root) = 0.837. 
Next, let us explain the case of Figure 4. Cal- 
culating the same way as Figure 3, the interactive 
disambiguation is executed in any node besides leaf 
ones, and P/,t, (root), P~,to (root) are a.s follows: 
Pi,,~. (root) 
P~,,to( ,'oot ) 
= 0.9(Pr(star_l) + Pr(star_2)) 
= 0.9(Pi, tr(s'car_l) + Pint~(star_2)) 
= 0.9(0.765+0.135) : 0.81 
= max(Pr(star_l), Pr(sl;ar_2)) 
= max(0.10,0.75) = 0.75 
Therefore, P,,t~(root) > P~u,o(rOot), and 
P,.(root) becomes 0.81. Comparing with P~(root) 
of each order, P~(root) of Figure 3 is greater than 
that of Figure 4. Thus the system interacts with 
users against 'see' in the first, place. 
5 Experiments 
We applied the proposed method(abbreviated as 
MP) to the disambiguation of trees of ca lldidates 
that are made for experiments, and compared it with 
the method (abbreviated as MA) that executes in- 
teraction in all nodes. 
We set the following properties to the tree of can- 
didates. 
• the number of ambiguous words included in an 
input sentence 
• the mlmber of meanings in an ambiguous word 
• the occurrence probability of candidates 
To assign an occurrence probability to each can- 
didate, a raudom value is given to each candidate 
above all, and each value is divided by the sum of 
values given to all candidates. 
Figure 5, 6 show the accuracy at the root node 
and the number of interaction, respectively. In these 
figures, a mark '+' indicates results of MI ). Each of 
them is the average of 300 trees. A mark "*" indicates 
results of MA. Because MA does not prescribe the 
order of disambiguation, the result of each tree is 
the average of all the orders. 
og 
o as 
oe 
~ o 75 
o7 
o 65 
A3. A~ A~. o3. a3~ ~. e4. C3. C~ CS- C,~* 03. Oa* D6. Oe. E6 ES* EIZ. EI2. r~. F6. 
I~optmy a tr~ 
Figure 5: The accuracy of MP, MA 
The horizontal axis means the property of the tree. 
Each Alphabet in the value of the horizontal axis 
stands for the number of ambiguous words in a tree 
and the nunlber of meanings of a word as follows: 
A: 2x4 D: 2x4x4 
B: 2x2x4 E: 2x2x4x4 
C: 2x2x2x4 F: 2x2x2x4x4 
1426 
• t 
4 
i I ~ ~i 
, i i i i i i , i , , i i i i i i , i L i i 
Aa Aa. Aa. A4. a3 ~, 84- B4. ca- Ca. C6. CS, Oa. 03. t)~ t~. Ee ES* E12 El2* e6 r~ 
Figure 6: The nurnber of interaction of MP, MA 
For instance, '2 x 4' shows that there are two am- 
biguous words ill a tree and one ambiguous word has 
two meanings and another word has four meanings. 
The lmmber in the value of the x-axis represents 
the number of the candidate whose occurrence prob- 
ability is not zero. Two marks, "+' and '-' mean that 
the accuracy of interactioll is 0.9, 0.85 respectively. 
6 Discussion 
6.1 The Accuracy of Disambiguation 
The effect of the proposed method on tile accuracy 
is expressed by the difference of distributions of two 
lnarks, '+' and '*' in Figure 5. This shows that the 
accuracy of the proposed method is better t.hall that 
of MA in ally property of tree. Table 1 (the line of 
"Accuracy') shows the minimum, maxinmln, and av- 
erage values of the ratio of ilnproved accuracy (RIA). 
The definition of RIA is shown as follows: 
RIA - acp - aCa 
1.0 - aca 
where acp, ac a is t.he accuracy the result by MP 
and MA respectively. 
Table 1: Summary of the results 
Minimum Maximuna Average 
Accuracy 0.14 0.23 0.18 
Interaction -0.06 0.12 0.03 
6.2 The Number of Interaction 
Tile number of interaction may increase on the con- 
dition that the accuracy of the analyzed result is 
maxinfized. Ill this section, the degree of the in- 
crease will be estimated by comparing the number 
of interaction of MP with that of MA. For this 
purpose, 'RII' is defined as follows: 
RII -np - na 
nw 
where np, na is the number of interaction by MP 
and MA respectively, 71.,,, is the llumber of ambigu- 
ous words in an input sentence. RII represents the 
ratio of the increase ill the number of interaction per 
ambiguous word. Table l(the lille of 'Interaction') 
shows the rnininaum, lnaximuna, and average of RII. 
To reduce the number of interaction, the auto- 
matte disambiguation is executed instead of execut- 
ing tile interactive disambiguation, estimating the 
loss of the accuracy L(i) ill node i. L(i) is defined 
as follows: 
L(i) = P,.(i)- Pat, to(i) 
The proposed lnethod will allow the system to re- 
duce the nunfi)er of interaction, by considering L(i) 
ill each node. 
7 Conclusion 
We have proposed the lnethod of combining the 
interactive disalnbiguation and the autonlatic one. 
The characteristic of our method is that it. considers 
the accuracy of the interactive disambiguat ion. This 
method makes three following things possible: 
• selecting the disambiguation method that ob- 
tains higher accuracy 
• limiting exl)lanations shown to users 
• obtaining the order of disaml)iguation where t he 
accuracy of the analyzed resuhs is maximized. 

References 
Herve' Blanchon, K. Loken-Kina, and T. Morimoto. 
1995. An interactive disambiguation module for 
English natural language utteracalwes. In Pro- 
ce¢dings of NLPRS"95, pages 550-555. 
Ralf D.Brawn and Sergei Nirenburg. 1990. Humall- 
computer interaction for semantic disambigua- 
tion. Ill Proccedings of COLING-90, pages 42-47. 
II. Maruyama and H. Watanabe. 1990. All interac- 
tive Japanese parser for machine trallslation. Ill 
Proceedings of COLING-90, pages 257-262. 
K. Muraki, S. Akamiue, K. Satoh, and S. Ando. 
1994. TWP: How to assist English production 
on Japanese word processor. Ill Proceedings of 
COLING-94, pages 847-852. 
K. Nagao and H. Maruyama. 1992. Ambiguities and 
their resolution in natural language processing. 
Journal of 1PSJ, 33(7):741-745. 
M. Yamaguchi, N. Inui, Y. Kotani, and H. Nisimura. 
1995. The design and experimem of all evaluation 
function for user interaction cost ill the interac- 
tive semantic disambiguation. Ill Proceedings of 
HCI'95, pages 285-290. 
