Proceedings of the Interactive Question Answering Workshop at HLT-NAACL 2006, pages 41–48,
New York City, NY, USA. June 2006. c©2006 Association for Computational Linguistics
Answering questions of Information Access Dialogue (IAD) task
using ellipsis handling of follow-up questions
Junichi Fukumoto
Department of Media Technology
Ritsumeikan University
1-1-1 Nojihigashi, Kusatsu, Shiga 525-8577 Japan
fukumoto@media.ritsumei.ac.jp
Abstract
In this paper, we propose ellipsis han-
dling method for follow-up questions in
Information Access Dialogue (IAD) task
of NTCIR QAC3. In this method, our sys-
tem classi es ellipsis patterns of question
sentences into three types and recognizes
elliptical elements using ellipsis handling
algorithm for each type. In the evalua-
tion using Formal Run and Reference Run
data, there were several cases which our
algorithm could not handle ellipsis cor-
rectly. According to the analysis of evalu-
ation results, the main reason of low per-
formance was lack of word information
for recognition of referential elements. If
our system can recognize word meanings
correctly, some errors will not occur and
ellipsis handling works well.
1 Introduction
In question answering task QAC of NTCIR (Kato
et al., 2005)(Kato et al., 2004), interactive use of
question answering is proposed as one of evaluation
task called Information Access Dialogue (IAD) task,
which was called subtask3 in QAC1,2. In IAD task,
a set of question consists of one  rst question and
several follow-up questions. These series of ques-
tions and answers comprise an information access
dialogue. In QAC1, there was only one follow-up
question in a series of questions, but in QAC2 and 3
there were several follow-up questions.
All follow-up questions have anaphoric expres-
sions including zero anaphora which is frequently
occurs in Japanese. There were several approaches
to answer follow-up questions. One approach was
to extract answers of follow-up questions from doc-
uments which were retrieved using clue words of the
 rst question (Sasaki et al., 2002). In the other ap-
proach, they added clue words extracted from the
previous questions to clue words of follow-up ques-
tion for document retrieval (Murata et al., 2002).
However, when topic was changed in a series of
questions, these approaches did not work well be-
cause clue words of the previous questions were
not always effective to extract answer of the current
question.
Our approach is to handle ellipses of follow-up
questions and apply the processed questions to ordi-
nary question answering system which extracts an-
swers of a question (Fukumoto et al., 2002)(Fuku-
moto et al., 2004)(Matsuda and Fukumoto, 2005).
For QAC3, we have improved our previous approach
to handle follow-up questions, that is, we have ex-
panded ellipsis handling rules more precisely. Based
on the analysis of evaluation results of QAC2, we
have classi ed ellipsis pattern of question sentences
into three types. The  rst type is ellipsis using pro-
noun. This is the case that a word used in previ-
ous questions is replaced with pronoun. The second
type is ellipsis of word in verb’s obligatory case el-
ements in the follow-up question. Some obligatory
case elements of a verb of a follow-up question will
be omitted and such elements also used in the previ-
ous question. The last type is ellipsis of a modi er
or modi cand in a follow-up question. Such an ele-
41
ment appears in the previous question and has mod-
i cation relationship with some word in the follow-
up question sentence. In order to handle the above
three ellipsis types, we utilized case information of
main verb of a question and co-occurrence of nouns
to recognize which case information is omitted. We
used co-occurrence dictionary which was developed
by Japan Electric Dictionary Research Inc. (EDR)
(EDR, ).
As for core QA system which is our main ques-
tion answering system, we have integrated previous
systems modules which are developed for QAC2.
One module is to handle numeric type questions. It
analyzes co-occurrence data of unit expression and
their object names and detects an appropriate nu-
meric type. Another module uses detailed classi ca-
tion of Named Entity for non numerical type ques-
tions such as person name, organization name and so
on to extract an answer element of a given question.
In the following sections, we will show the de-
tails of analysis of elliptical question sentences and
our new method of ellipsis handling. We will also
discuss our system evaluation on ellipsis handling.
2 Ellipsis handling
In this section, we explain what kinds of ellipsis pat-
terns exist in the follow-up questions of a series of
questions and how to resolve each ellipsis to apply
them to core QA system.
2.1 Ellipsis in questions
We have analyzed 319 questions (46sets) which
were used in subtask3 of QAC1 and QAC2 and then,
classi ed ellipsis patterns into 3 types as follows:
Replacing with pronoun
In this pattern, pronoun is used in a follow-up ques-
tion and this pronoun refers an element or answer of
the previous question.
Ex1-1 a0a2a1a4a3a6a5a8a7a10a9a12a11a2a13a15a14a17a16a19a18a12a20a2a21a23a22
(Who is the president of America?)
Ex1-2 a24a26a25a28a27a17a29a31a30a33a32a35a34 a7a12a14a12a36a38a37a8a18a38a20a39a21a40a22
(When did it become independent?)
In the above example, pronoun  a24a4a25 (it) of
question Ex1-2 refers a word  a0a31a1a15a3a6a5 (America) 
of question Ex1-1. The question Ex1-2 should be  
a0a12a1a41a3a42a5
a27a43a29a38a30a44a32a45a34
a7a23a14a40a36a40a37a12a18a38a20a31a21a46a22 (When does
America become independent?) in a completed
form.
Ex2-1 a0a2a1a4a3a6a5a8a7a10a9a12a11a2a13a15a14a17a16a19a18a12a20a2a21a23a22
(Who is the president of America?)
Ex2-2 a47 a7a39a48a35a49a12a50a41a14a52a51 a25 a18a12a20a2a21a23a22
(Where is his birth place?)
In the above example, pronoun  a47 (his) of ques-
tion Ex2-2 refers an answer word  a53a55a54a17a56a31a57 (J.
Bush) of question Ex2-1. The question Ex2-2
should be  a53a40a54a58a56a35a57 a7a23a48a59a49a43a50a26a14a55a51 a25 a18a17a20a38a21a43a22 (Where
is J. Bush’s birth place?) in a completed form.
Ellipsis of an obligatory case element of verb
In this pattern, an obligatory case element verb in
follow-up question is omitted, and the omitted el-
ement refers an element or answer of the previous
question. An example of this pattern is as follows:
Ex3-1 a0a2a1a4a3a6a5a8a7a10a9a12a11a2a13a15a14a17a16a19a18a12a20a2a21a23a22
(Who is the president of America?)
Ex3-2 a36a38a37a46a60a31a61 a32a43a62a63a32a28a34 a21a23a22
(When did φ inaugurate?)
In the above example, the verb  a60a63a61a41a20a4a64 (in-
augurate) has two obligatory case frames  agent 
and  goal , and the elements of each case frame are
omitted. The element of  agent is the answer of
Ex3-1, and the element of  goal is  a9a19a11a19a13 (the
President) of Ex3-1. Therefore, Ex3-2 should be
 (the answer of Ex3-1) a14a38a36a40a37a46a9a23a11a23a13a15a65a35a60a12a61 a32a28a62a55a32
a34
a21a46a22 (When did (the answer of Ex3-1) inaugurated
as the President?) .
Ellipsis of a modi er or modi cand
This pattern is the case of ellipsis of modi er. When
there is modi cation relation between two words of
a question, either of them (modifying element or the
modi ed element) modi es an element of the next
question but is omitted. We call the modifying el-
ement modi er and we call the modi ed element
modi cand. The following example shows ellipsis
of modi er.
Ex4-1 a0a2a1a4a3a6a5a8a7a10a9a12a11a2a13a15a14a17a16a19a18a12a20a2a21a23a22
(Who is the president of America?)
Ex4-2 a66a10a67a2a68a15a69 a14a28a16a19a18a12a20a2a21a23a22
(Who is a minister of state?)
In the above example, the word  a0a23a1a55a3a58a5 (Amer-
ica) is modi er of  a9a39a11a8a13 (the president) in the
question Ex4-1. Then, the word  a0a19a1a4a3a70a5 (Amer-
ica) also modi es  a66a43a67a2a68a52a69 (a minister of state) 
42
of Ex4-2 and is also omitted. The question Ex4-2
should be  a0a12a1a15a3a58a5a23a7 a66a45a67a38a68a26a69 a14a71a16a8a18a46a20a23a21a72a22 (Who
is a minister of state of America?) .
The following example shows ellipsis of modi -
cand.
Ex5-1 a0a2a1a4a3a6a5a8a7a10a9a12a11a2a13a15a14a17a16a19a18a12a20a2a21a23a22
(Who is the president of America?)
Ex5-2 a73a12a74a63a75a45a76
a14a28a16a19a18a12a20a2a21a23a22
(Who is φ of France?)
In this example, the word  a9a40a11a46a13 (the president) 
is modi cand of the word  a0a39a1a77a3a59a5 (America) in
the question Ex5-1. In the question Ex5-2, the word
 a73a8a74a55a75a35a76 (France) should modi es the word  a9
a11a2a13 (the president) which is omitted in the ques-
tion Ex5-2. Then the question Ex5-2 should be  a73
a74a39a75a35a76
a7a35a9a23a11a23a13a15a14a28a16a39a18a40a20a2a21a46a22 (Who is the president
of France?) .
2.2 How to resolve ellipsis
2.2.1 Overview of the method
We will show ellipsis resolution method of these
three patterns. For the  rst pattern, we replace the
pronoun with a word which referred by it. For
the second pattern, we try to  ll up obligatory case
frames of the verb. For the third pattern, we take
a word from the previous question based on co-
occurrence frequency. We assumed that the an-
tecedent of an elliptical question exists in a question
which appears just before, so the  previous ques-
tion indicates immediately previous question in our
method. We show the process as follows:
Step1 Estimate the pattern of ellipsis:
When a follow-up question has pronoun, this is
the case of the  rst pattern. When a follow-up
question has some verb which has an omitted
case element, this is the case of the second pat-
tern. When a follow-up question has no pro-
noun and such a verb, this is the case of the
third pattern.
Step2 Estimate kinds of the omitted word:
Step2a When the ellipsis pattern is the  rst pattern:
Estimate the kind of word which the pronoun
refers. When the pronoun directly indicates
kinds of word (ex: a47 : he), depend on it. If
the pronoun does not directly indicate kinds of
word (ex: a24 a7 :its +noun), use the kind of the
word which exists just behind the pronoun.
Step2b When the ellipsis pattern is the second pat-
tern:
Estimate obligatory case frame of the verb of
the follow-up question. Then, estimate omitted
element of the case frame and the type of the
element.
Step2c When the ellipsis pattern is the third pattern:
Get a noun X which appears with Japanese
particle  a14 (ha) 1 in the follow-up question.
When compound noun appears with  a14 (ha) ,
the last word is assumed to be X. Then, col-
lect words which are modi er or modi cand
of X from corpus. If the same word as col-
lected words is in the previous question, take
over the word and skip step3. Otherwise, esti-
mate the kind of word which is suitable to mod-
i er (or modi cand) of X. Estimate the kind of
collected modi ers and modi cands, and adopt
one which has the highest frequency.
Step3 Decide the succeeded word of the previous
question:
Estimate type of answer of previous question 2
and kind of each word used in previous ques-
tion from rear to front. When a word has a kind
 t for the estimate in step2, take the word to
follow-up question.
2.2.2 EDR thesauruses dictionary
We have used thesauruses of EDR dictionary to
estimate the kind of words, obligatory case frame of
verbs, omitted element of case frame, and to collect
modi er and modi cand of a word. Details are as
follows:
Estimation of word type
We used EDR Japanese Word Dictionary and
EDR Concept Dictionary. Japanese Word Dictio-
nary records Japanese words and its detailed concept
as Concept Code, and Concept Dictionary records
each Concept Code and its upper concept. We check
a target word using Japanese Word Dictionary and
1This particle is used as topic marker in Japanese.
2Use core QA’s module
43
get its detailed concept code. Then, we generalize
type of the word using concept code of Concept Dic-
tionary.
For example, concept code of a word  a78a12a79 (com-
pany) is 3ce735 which means  a group of people
combined together for business or trade . We will
check its upper concept using Concept Dictionary,
for example, upper concept of 3ce735 is 4449f5, up-
per concept of 4449f5 is 30f74c, and so on. Finally,
we can get word type of 3ce735 as 3aa912 which
means  agent (self-functioning entity) . Therefore,
we can estimate that type of word  a78a23a79 (company) 
is an agent.
Estimation of obligatory case frame of verb and
omitted element
We will use EDR Japanese Cooccurrence Dic-
tionary for estimation of omitted case element.
Japanese Cooccurrence Dictionary contains infor-
mation of verb case frame and concept code with
Japanese particle for each case. We will check oblig-
atory case frame and omitted element. Firstly, we
check a verb with Japanese Cooccurrence Dictio-
nary and get its case frame, concept code and par-
ticle information. Then we can recognize omitted
case element by particle information and estimate
word type of omitted element.
For example, according to the Japanese Cooc-
currence Dictionary, a verb  a60a15a61a4a20a80a64 (inaugu-
rate) has two case frames, agent (30f6b0) and goal
(3f98cb or 3aa938), and agent is used with particle  
a27 (ga) , goal is used with particle  
a65 (ni) . If ques-
tion doesn’t have any  a27 (ga) or  a65 (ni) (ex:  a36
a37a10a60a23a61
a32a45a62a39a32a59a34
a21a72a22 (When did φ inaugurate?) ), we
estimate that agent and goal are omitted. Then, we
estimate kind of the omitted element same as  Esti-
mation of kind of words .
Collection of modi er and modi cand
Japanese Cooccurrence Dictionary contains
Japanese co-occurrence data of various modi -
cations. We will use the co-occurrence data to
collect modi er or modi cand of word X. Details as
follows:
1. Search  X a7 (no) noun (noun of X) and  noun
a7 (no) X (X of noun) pattern from Japanese
Cooccurrence Dictionary
2. When Y appears in the  Y a7 (no) X (X of Y) 
pattern, we can estimate Y as modi er of X.
3. When Y appears in the  X a7 (no) Y (Y of X) 
pattern, we can estimate Y as modi cand of X.
2.2.3 Examples of ellipsis handling
We will show above examples of ellipsis handling
in the following.
Example of ellipsis handling of  rst pattern3
Ex1-1 a0a2a1a4a3a6a5a8a7a10a9a2a11a12a13a52a14a28a16a63a18a38a20a39a21a40a22
(Who is the president of America?)
Ex1-2 a24a26a25a28a27a17a29a31a30a33a32a28a34 a7a2a14a38a36a12a37a31a18a12a20a2a21a23a22
(When did it become independent?)
Ex1-2’ a0a2a1a4a3a6a5 a27a17a29a31a30a33a32a28a34 a7a2a14a38a36a12a37a31a18a12a20a2a21a40a22
(When did America become independent?)
In the above example, Ex1-2 has a pronoun  a24a39a25
(it) , so we classi ed ellipsis pattern of Ex1-2 into
the  rst pattern. Pronoun  a24a55a25 (it) refers organi-
zation or location by information of pronoun. The
word  a0a19a1a4a3a59a5 (America) has information of lo-
cation but the word  a9a31a11a31a13 (the president) are not
organization or location. Then we can estimate that
pronoun  a24a26a25 (it) of Ex1-2 refers the word  a0a8a1
a3a59a5 (America) of Ex1-1. Question Ex1-2 should
be  a0a63a1a81a3a28a5a26a7a46a9a19a11a19a13a82a14a40a16a41a18a39a20a55a21a2a22 (Who is the
president of America?) .
Example of ellipsis handling of second pattern
Ex3-1 a0a2a1a4a3a6a5a8a7a10a9a2a11a12a13a52a14a28a16a63a18a38a20a39a21a40a22
(Who is the president of America?)
Ex3-2 a36a38a37a46a60a31a61 a32a43a62a15a32a35a34 a21a40a22
(When did he inaugurated?)
Ex3-2’ (answer of Ex3-1) a14a23a36a23a37a46a9a31a11a31a13a52a65
a60a31a61
a32a43a62a15a32a35a34
a21a40a22
(When did (answer of Ex3-1) inaugurated?)
In the above example, Ex3-2 has a verb  a60a39a61a63a20
a64 (inaugurate) , so we classi ed ellipsis pattern of
Ex3-2 into the second pattern. The word  a60a19a61a26a20
a64 (inaugurate) has two obligatory case: agent (hu-
man) and goal (managerial position). Ex3-2 doesn’t
have word which is suitable for obligatory cases of  
a60a31a61a26a20a41a64 (inaugurate) . Therefore we estimate that
the agent and the goal are omitted. Then, we esti-
mate answer type of Ex3-1 and kind of each word
of Ex3-1. The answer type of Ex3-1 is human, so it
3Exm-n’ indicates complemented question of Exm-n
44
is suitable for the agent. The kind of  a9a39a11a2a13 (the
president) is managerial position, so it is suitable
for the goal. Finally, we take the answer of Ex3-
1 and  a9a31a11a8a13 (the president) to Ex3-2 and Ex3-2
becomes  (answer of Ex3-1) a14a2a36a31a37a38a9a2a11a8a13a4a65a28a60a39a61
a32a28a62a55a32a35a34
a21a38a22 (When did (answer of Ex3-1) inaugu-
rated?) .
Example of ellipsis handling of third pattern
Ex4-1 a0a2a1a4a3a6a5a8a7a10a9a2a11a12a13a52a14a28a16a63a18a38a20a39a21a40a22
(Who is the president of America?)
Ex4-2 a66a10a67a2a68a15a69 a14a28a16a63a18a38a20a39a21a40a22
(Who is a minister of state?)
Ex4-2’ a0a2a1a4a3a6a5a8a7 a66a43a67a12a68a52a69 a14a17a16a19a18a12a20a2a21a23a22
(Who is a minister of state of America?)
In the above example, Ex4-2 doesn’t have any
pronoun and verb, so we classi ed ellipsis pattern of
Ex4-2 into the third pattern. Then we search  noun
a7
a66a43a67a2a68a41a69 (a minister of noun) and  a66a43a67a31a68a41a69
a7
noun (noun of a minister) pattern from the Japanese
Cooccurrence Dictionary. In the Japanese Cooccur-
rence Dictionary, we can  nd  a0a12a1a41a3a42a5a23a7 a66a45a67a23a68a63a69
(a minister of America) pattern.  a0a8a1a41a3a6a5 (Amer-
ica) is used in Ex4-1, so we take over  a0a26a1a83a3a45a5
(America) to Ex4-2 and Ex4-2 becomes  a0a39a1a82a3
a5a8a7
a66a17a67a8a68a77a69
a14a17a16a55a18a31a20a19a21a38a22 (Who is a minister of
state of America?) .
3 Evaluation
3.1 Evaluation method
We have evaluated our QA system only on ellipses
handling. The following example shows question
sets of the Formal Run and Reference Run. In Qm-
n, m and n indicates series ID and its question num-
ber which we gave and Rm-n indicates a question
which correspond to Qm-n.
Questions of Formal Run
Q1-1 a84a23a85a52a86a83a87a70a88a26a89a46a88
a14a38a36a12a37a46a90a15a91a63a92a10a93
a62a55a32a10a34
a21a38a22
(When was Mt.Fuji radar installed?)
(QAC3-30038-01)
Q1-2 a51a8a94a70a36a81a94a38a95a17a96a72a18a40a90a63a91a15a92a45a93 a62a63a32a28a34 a21a38a22
(What kind of purpose was it installed by?)
(QAC3-30038-02)
Q1-3 a84a23a85a52a86 a7a10a97a8a98a15a65a31a99a82a100 a62a63a32a28a34 a21a23a22
(Which area of Mt.Fuji was it installed?)
(QAC3-30038-03)
Q1-4 a51a10a7a63a101a63a94a70a102a17a103a12a104a81a105a59a106a77a107 a62a15a32a35a34 a21a40a22
(What kind of award did it get?)
(QAC3-30038-04)
Questions of Reference Run
R1-1 a84a23a85a52a86a83a87a70a88a26a89a46a88 a14a38a36a12a37a46a90a55a91a55a92a35a93 a62a55a32a10a34 a21a38a22
(When was Mt.Fuji radar installed?)
(QAC3-31267-01)
R1-2 a84a23a85a52a86a83a87a70a88a26a89a46a88 a14a52a51a8a94a70a36a82a94a23a95a43a96a46a18a46a90a55a91a55a92
a93
a62a63a32a28a34
a21a23a22 (What kind of purpose was
Mt.Fuji radar installed by?)
(QAC3-31268-01)
R1-3 a84a23a85a52a86a83a87a70a88a26a89a46a88 a14 a84a23a85a52a86 a7a28a97a2a98a41a65a12a99a81a100 a62
a32a28a34
a21a23a22 (Which area of Mt.Fuji was Mt.
Fuji radar installed?)
(QAC3-31269-01)
R1-4 a84a23a85a52a86a83a87a70a88a26a89a46a88 a14a52a51a10a7a63a101a26a94a59a102a43a103a31a104a82a105a71a106a52a107
a62a15a32a35a34
a21a40a22 (What kind of award did Mt.
Fuji radar get?)
(QAC3-31270-01)
In IAD task, one series of questions consists of the
 rst question and several follow-up questions which
contain ellipsis. In our current implementation, we
assumed that antecedent of an elliptical question ex-
ists in its just before question. For example, the
antecedent of Q1-2 is  a84a8a85a108a86a80a87a10a88a15a89a8a88 (Mt.Fuji
radar) of Q1-1. The antecedent of Q1-4 is  a84a31a85
a86a77a87a71a88a39a89a40a88 (Mt.Fuji radar) of Q1-1 actually, how-
ever, if Q1-3 is completed correctly (as R1-3),  a84a72a85
a86a77a87a71a88a26a89a40a88 (Mt.Fuji radar) exists in Q1-3. There-
fore, we prepared evaluation data from QAC test set,
310 pairs of questions. One pair consists of a ques-
tion of Reference Run and a question of Formal Run.
For example, R1-1 and Q1-2 is one pair of the eval-
uation data, R1-3 and Q1-4 is other one. We have
evaluated our method using this data. Correctness
has been judged by human. When the system must
take an answer of previous question, we have used
45
 <ANS> which indicates the answer of previous
question. 4
3.2 Results
Our system could complete 52 of 310 questions cor-
rectly as results. 28 among 52 success cases are
done by ellipsis handling method proposed in the
previous QAC evaluation. Our previous approach
is based on topic presentation in question sentences.
If there is an ellipsis in a question, we will use infor-
mation of topic information in the previous question.
Topic presentation is detected by Japanese particle  
a14 (ha) . The other cases of 24 were succeeded by
the approach described above. We will show the de-
tails as follows:
• Replacing with pronoun:
System classi ed 88 of 310 questions in this
pattern. The all of 88 classi cations were cor-
rect. 12 of 88 questions were completed cor-
rectly.
• Ellipsis of an obligatory case element of verb:
System classi ed 158 of 310 questions as this
pattern. 105 of 158 classi cations were correct.
8 of 105 questions were completed correctly.
• Ellipsis of a modi er or modi cand:
System classi ed 64 of 310 questions as this
pattern. 44 of 64 classi cations were correct. 4
of 44 questions were completed correctly.
Major failure cases and their numbers which are
indicated with dots are as follows:
Failure of classi cation of ellipsis pattern
• System uses wrong verbs...29
• All obligatory cases of verb is  lled and other
element is omitted...22
• Failure of morphological analysis...8
• An adjective phrase is omitted...1
4In the Formal Run, we have replace  <ANS> with the
1st answer of core QA. In the evaluation, considering core QA’s
failure, we have left  <ANS> and considered as correct.
Failure of estimation of omitted element of
follow-up question
• Verb isn’t recorded in Japanese Cooccurrence
Dictionary...35
• Shortage of rules for pronoun...17
• System  lls up to case already  lled up...15
• Any modi er or modi cand doesn’t exist in
Japanese Cooccurrence Dictionary...10
• Case frame element is omitted but system fails
to  nd it...7
• Verb is passive voice...6
• System fails to select the element of modi ca-
tion relation...6
• Question doesn’t have element of case frame
and it is unnecessary...2
Failure of decision of which word should be
taken
• System fails to estimate word type of answer in
the previous question...79
• System fails to decide to scope of target
word...21
• A modi er or modi cand which has lower co-
occurrence frequency should be taken...7
• System takes inappropriate word from an inter-
rogative phrase...6
• Answer type of the previous question has same
kind with a word should be taken...3
4 Discussion
Our system could work well for some elliptical ques-
tions as described in the previous section. We will
show some examples and detail of major failure
analysis results in the following.
1. Verb case elements:
There was a Japanese delexical verb5  a36a39a64  in
a follow-up question, then our system could not
5Delexical verb is a functional verb which has speci c
meaning in it.
46
 ll up its obligatory cases because every oblig-
atory cases of this verb had already  lled up.
It is necessary to handle these delexical verbs
such as  a36a15a64  ,  a102a15a64  ,  a36a108a94  and so on as
stop words.
Otherwise, there were several questions in
which all obligatory cases of verb has already
 lled up. In this case, it is necessary to ap-
ply the other approach. In the example  
a109
a88a55a110a52a75a71a111a113a112
a7
a73a8a114a35a115a15a116
a5
a54a4a117a119a118
a65a26a48a43a120
a32
a34a23a121a15a122
a14a12a16a4a18
a32a38a34
a21a19a22 (What is the actor’s
name who attended opening event in the  rst
day?) , some additional information for  open-
ing event is omitted. Moreover, there were
some verbs which had no case information in
EDR dictionary. It would be helpful to check
co-occurrence with this word in the previous
question.
2. Morphological analysis failure:
The expression  a24a15a25 a18 (sokode) in question
sentence was recognized as one conjunction  
a24a55a25
a18 (then) although it should be analyzed
in  a24a55a25 (soko: there) +  a18 (de: at) . If mor-
phological analyzer works well, our algorithm
could handle ellipsis correctly.
3. Lack of rules for pronoun:
In the expression  a25
a7a35a123a63a124
a76a40a125a8a88a12a56a15a126a71a75 (this
space station) of question sentence, ellipsis
handling rule for pronoun  a25 a7 (this) was not
implemented, then our method could not han-
dle this case. It is necessary to expand our al-
gorithm for this case.
4. case information handling error:
q1 a127a38a128a12a129a2a130a23a131a15a27a19a132a8a133a17a76a55a134a40a88 a105 a32a71a135 a36 a34
a7a2a14a52a51
a25
a7
a125a108a87a17a136a6a137
a18a23a20a8a21a38a22 (Which
TV station is Ms. Sawako Agawa
working as TV caster?) (QAC3-31206-01)
q2 a111a15a138a23a135a72a139 a36 a34a35a68a2a140a31a141a2a142 a14a28a97a63a18a23a20a8a21a38a22
(What is the title of long novel which φ
 rstly wrote?) (QAC3-30029-05)
In the above example (q1 is the  rst question
and q2 is follow-up question), system checks
obligatory case elements of verb  a139a144a143 (write) 
of question q1. The verb  a139a145a143  has three
obligatory cases: agent, object and goal ac-
cording to EDR dictionary. System estimated
that every obligatory case element were omit-
ted, and checks  a127a2a128a26a129a26a130a19a131 (Ms. Sawako
Agawa) ,  a132a8a133a17a76a41a134a23a88 (TV caster) ,  a132a39a133a72a76
a134a23a88 (TV caster) respectively. However, ob-
ject case of verb  a139a146a143  was  a68a39a140a8a141a8a142 (long
novel) of question q2 actually. In this ques-
tion, this element was modi ed by verb  a139a147a143
(write) , then system failed to estimate that the
object was already  lled. So, our algorithm
tried to  ll this object case up as  a132a39a133a17a76a15a134a23a88
(TV caster) . It is necessary to improve pat-
terns of estimation of omitted case element.
5. lack of co-occurrence information:
q3 a112a58a148a12a149a8a150a63a151 a7a10a152a2a9a23a153a77a14a35a154a2a155a55a36a12a37a72a156a26a157
a93a41a64a40a7a23a18a38a20a39a21a40a22 (When is Reitaisai of
Nikko Toshogu held in every year?)
(QAC3-31235-01)
q4 a158a55a159a39a74a38a159a160a117 a14a10a97a55a18a38a20a39a21a40a22 (What is the
highlight?)(QAC3-30033-06)
q4’ a112a58a148a12a149a8a150a63a151 a7 a158a55a159a39a74a38a159a160a117 a14a10a97a55a18a38a20a39a21a40a22
(What is the highlight of Nikko Toshogu?)
In the above example, q3 is the  rst question
and q4 is the follow-up question. The ques-
tion q4 is replaced with q4’ using ellipsis han-
dling. In this case, system took wrong mod-
i er  a112a70a148a63a149a63a150a4a151 (Nikko Toshogu) for  a158
a159a41a74a2a159a161a117 (highlight) . It is caused by lack
of co-occurrence information in EDR Japanese
Cooccurrence Dictionary because these words
are proper nouns which are not frequently used.
In order to handle such cases, it is necessary to
use co-occurrence information using large cor-
pus.
6. Passive verb expression:
In our current implementation, our system has
no rule to handle passive verb. In case of pas-
sive voice, it is necessary to check other case
element for ellipsis handling.
7. Multiple candidates:
47
q5 a162 a3 a75a164a163a166a165a81a167a35a168a23a115 a14a17a16a15a65 a66a10a67a2a68a15a69 a65
a169a63a170
a92a10a93
a34
a7a40a18a12a20a2a21a23a22 (Who appointed
Mr. Collin Powell as a minister of state?)
(QAC3-31087-01)
q6 a47 a7a35a171a8a172a52a96a38a102 a30a2a173 a14a4a51a45a7a15a101a19a94a71a102a15a174a72a7
a18a12a20a2a21a23a22 (What is his political situation?)
(QAC3-30013-03)
q6’ <ANS> a7a10a171a2a172a77a96a40a102 a30a12a173 a14a52a51a10a7a63a101a63a94
a102a52a174a43a7a23a18a38a20a39a21a40a22 (What is <ANS>’s
political situation?)
In the above example, q5 is the  rst question
and q6 is the follow-up question. The question
q6 is replaced with q6’ using ellipsis handling
rules. System replaced  a47 (his) of q6 with the
answer of q5. Because  a47 (his) refers human
and the answer type of q5 is human, and the an-
swer of q5 was the nearest word which suitable
to  a47 (his) . But,  a47 (his) referred  a162 a3 a75a41a163
a165a82a167a10a168a23a115 (Mr. Colin Powell) actually. In this
case,  a162 a3 a75a164a163a166a165a81a167a10a168a23a115 (Mr. Colin Powell) 
was the topic of q5, so  a162 a3 a75a55a163a175a165a77a167a35a168a72a115 (Mr.
Colin Powell) would be better one than the an-
swer of q5. Topic information handling would
be implemented in our algorithm.
5 Conclusion
In this paper, we have presented ellipsis handling
method for follow-up questions in IAD task. We
have classi ed ellipsis pattern of question sentences
into three types and proposed ellipsis handling al-
gorithm for each type. In the evaluation using For-
mal Run and Reference Run data, there were sev-
eral cases which our algorithm could not handle el-
lipsis correctly. According to the analysis of eval-
uation results, the main reason of low performance
was lack of word information for recognition of ref-
erential elements. If our system can recognize word
meanings correctly, some errors will not occur and
ellipsis handling works well.
We have already improved our ellipsis handling
method with recognition of target question. In the
evaluation of QAC3, our system searches elliptical
element in the previous question. However, we have
not tested this new algorithm using test correction.
In the future work, we will test this algorithm and
apply it for other QA application.

References
EDR Home Page
. http://www2.nict.go.jp/kk/e416/EDR/J index.html.
Junichi Fukumoto, Tetsuya Endo, and Tatsuhiro Niwa.
2002. Rits-QA: Ritsumeikan question answering sys-
tem used for QAC-1. In Working Notes of the 3rd NT-
CIR Workshop Meeting: Part IV QAC1, pages 113 
116. National Institute of Informatics.
Junichi Fukumoto, Tatsuhiro Niwa, Makoto Itoigawa,
and Megumi Matsuda. 2004. Rits-QA: List answer
detection and context task with ellipsis handling. In
Working Notes of the 4th NTCIR Workshop Meeting,
pages 310 314. National Institute of Informatics.
Tsuneaki Kato, Junichi Fukumoto, and Fumito Masui.
2004. Question answering challenge for information
access dialogue - overview of NTCIR-4 QAC2 subtask
3. In Proceesings of the 5th NTCIR Workshop Meet-
ing on Evaluation of Information Access Technologies,
pages 291 297. National Institute of Informatics.
Tsuneaki Kato, Junichi Fukumoto, and Fumito Masui.
2005. An overview of NTCIR-5 QAC3. In Pro-
ceesings of the 5th NTCIR Workshop Meeting on
Evaluation of Information Access Technologies, pages
361 372. National Institute of Informatics.
Megumi Matsuda and Junichi Fukumoto. 2005. Answer-
ing questions of IAD task using reference resolution of
follow-up questions. In Proceesings of the 5th NTCIR
Workshop Meeting on Evaluation of Information Ac-
cess Technologies, pages 414 421. National Institute
of Informatics.
Masaki Murata, Masao Utiyama, and Hitoshi Isahara.
2002. A question-answering system using unit estima-
tion and probabilistic near-terms ir. In Working Notes
of NTCIR Workshop 3 Meeting QAC1, pages 47 54.
National Institute of Informatics.
Yutaka Sasaki, Hideki Isozaki, Tsutomu Hirao, Koji
Kokuryou, and Eisaku Maeda. 2002. NTT’s QA sys-
tems for NTCIR QAC-1. In Working Notes of NTCIR
Workshop 3 Meeting QAC1, pages 63 70. National In-
stitute of Informatics.
