Proceedings of the Workshop on Sentiment and Subjectivity in Text, pages 39–46,
Sydney, July 2006. c©2006 Association for Computational Linguistics
Searching for Sentences Expressing Opinions 
by using Declaratively Subjective Clues 
 
 
 
Nobuaki Hiroshima, Setsuo Yamada, Osamu Furuse and Ryoji Kataoka 
NTT Cyber Solutions Laboratories, NTT Corporation 
1-1 Hikari-no-oka Yokosuka-Shi Kanagawa, 239-0847 Japan 
hiroshima.nobuaki@lab.ntt.co.jp 
 
 
 
Abstract 
This paper presents a method for search-
ing the web for sentences expressing 
opinions. To retrieve an appropriate 
number of opinions that users may want 
to read, declaratively subjective clues are 
used to judge whether a sentence ex-
presses an opinion. We collected declara-
tively subjective clues in opinion-
expressing sentences from Japanese web 
pages retrieved with opinion search que-
ries. These clues were expanded with the 
semantic categories of the words in the 
sentences and were used as feature pa-
rameters in a Support Vector Machine to 
classify the sentences. Our experimental 
results using retrieved web pages on 
various topics showed that the opinion 
expressing sentences identified by the 
proposed method are congruent with sen-
tences judged by humans to express 
opinions. 
1 Introduction 
Readers have an increasing number of opportu-
nities to read opinions (personal ideas or beliefs), 
feelings (mental states), and sentiments (positive 
or negative judgments) that have been written or 
posted on web pages such as review sites, per-
sonal web sites, blogs, and BBSes. Such subjec-
tive information on the web can often be a useful 
basis for finding out what people think about a 
particular topic or making a decision. 
A number of studies on automatically extract-
ing and analyzing product reviews or reputations 
on the web have been conducted (Dave et al., 
2003; Morinaga et al., 2002; Nasukawa and Yi, 
2003; Tateishi et al., 2004; Kobayashi et al., 
2004). These studies focus on using sentiment 
analysis to extract positive or negative informa-
tion about a particular product. Different kinds 
of subjective information, such as neutral opin-
ions, requests, and judgments, which are not ex-
plicitly associated with positive/negative as-
sessments, have not often been considered in 
previous work. Although sentiments provide 
useful information, opinion-expressing sentences 
like “In my opinion this product should be 
priced around $15,” which do not express ex-
plicitly positive or negative judgments (unlike 
sentiments) can also be informative for a user 
who wants to know others’ opinions about a 
product. When a user wants to collect opinions 
about an event, project, or social phenomenon, 
requests and judgments can be useful as well as 
sentiments. With open-domain topics, sentences 
expressing sentiments should not be searched 
exclusively; other kinds of opinion expressing 
sentences should be searched as well. 
The goal of our research is to achieve a web 
search engine that locates opinion-expressing 
sentences about open-domain topics on products, 
persons, events, projects, and social phenomena. 
Sentence-level subjectivity/objectivity classifica-
tion in some of the previous research (Riloff and 
Wiebe, 2003; Wiebe and Riloff, 2005) can iden-
tify subjective statements that include specula-
tion in addition to positive/negative evaluations. 
In these efforts, the subjectivity/objectivity of a 
current sentence is judged based on the existence 
of subjective/objective clues in both the sentence 
itself and the neighboring sentences. The subjec-
tive clues, some adjective, some noun, and some 
verb phrases, as well as other collocations, are 
learned from corpora (Wiebe, 2000; Wiebe et al., 
2001). Some of the clues express subjective 
meaning unrestricted to positive/negative meas-
urements. The sentence-level subjectivity ap-
39
proach suggests a way of searching for opinion 
expressing sentences in the open domain.  
The problem of applying sentence-level sub-
jectivity classification to opinion-expressing sen-
tence searches is the likelihood of collecting too 
many sentences for a user to read. According to 
the work of Wiebe et al. (2001), 70% of sen-
tences in opinion-expressing articles like editori-
als and 44% of sentences in non-opinion ex-
pressing articles like news reports were judged 
to be subjective. In analyzing opinions (Cardie 
et al., 2003; Wilson et al., 2004), judging docu-
ment-level subjectivity (Pang et al., 2002; Tur-
ney, 2002), and answering opinion questions 
(Cardie et al., 2003; Yu and Hatzivassiloglou, 
2003), the output of a sentence-level subjectivity 
classification can be used without modification. 
However, in searching opinion-expressing sen-
tences, it is necessary to designate criteria for 
opinion-expressing sentences that limit the num-
ber of retrieved sentences so that a user can sur-
vey them without difficulty. While it is difficult 
to formally define an opinion, it is possible to 
practically tailor the definition of an opinion to 
the purpose of the application (Kim and Hovy, 
2004).  
This study introduces the notion of declara-
tively subjective clues as a criterion for judging 
whether a sentence expresses an opinion and 
proposes a method for finding opinion-
expressing sentences that uses these clues. De-
claratively subjective clues such as the subjec-
tive predicate part of the main clause and subjec-
tive sentential adverb phrases suggest that the 
writer is the source of the opinion. We hypothe-
size that a user of such an “opinion-expressing 
sentence” search wants to read the writer’s opin-
ions and that explicitly stated opinions are pre-
ferred over quoted or implicational opinions. We 
suppose that writer’s ideas or beliefs are explic-
itly declared in a sentence with declaratively 
subjective clues whereas sentences without de-
claratively subjective clues mainly describe 
things. The number of sentences with declara-
tively subjective clues is estimated to be less 
than the number of subjective sentences defined 
in the previous work. We expect that the opinion 
expressing sentences identified with our method 
will be appropriate from the both qualitative and 
quantitative viewpoints. 
Section 2 describes declaratively subjective 
clues and explains how we collected them from 
opinion-expressing sentences on Japanese web 
pages retrieved with opinion search queries. Sec-
tion 3 explains our strategy for searching opin-
ion-expressing sentences by using declaratively 
subjective clues. Section 4 evaluates the pro-
posed method and shows how the opinion-
expressing sentences found by the proposed 
method are congruent with the sentences judged 
by humans to be opinions. 
2 Declaratively Subjective Clues 
Declaratively subjective clues are a basic crite-
rion for judging whether a sentence expresses an 
opinion. We extracted the declaratively subjec-
tive clues from Japanese sentences that evalua-
tors judged to be opinions. 
2.1 Opinion-expressing Sentence Judgment 
We regard a sentence to be “opinion expressing” 
if it explicitly declares the writer’s idea or belief 
at a sentence level. We define as a “declaratively 
subjective clue”, the part of a sentence that con-
tributes to explicitly conveying the writer’s idea 
or belief in the opinion-expressing sentence. For 
example, "I am glad" in the sentence "I am glad 
to see you" can convey the writer’s pleasure to a 
reader, so we regard the sentence as an “opinion-
expressing sentence” and “I am glad” as a “de-
claratively subjective clue.” Another example of 
a declaratively subjective clue is the exclamation 
mark in the sentence "We got a contract!" It con-
veys the writer’s emotion about the event to a 
reader. 
If a sentence only describes something ab-
stract or concrete even though it has word-level 
or phrase-level subjective parts, we do not con-
sider it to be opinion expressing. On the other 
hand, some word-level or phrase-level subjective 
parts can be declaratively subjective clues de-
pending on where they occur in the sentence. 
Consider the following two sentences. 
 
(1) This house is beautiful. 
(2) We purchased a beautiful house. 
 
Both (1) and (2) contain the word-level subjec-
tive part "beautiful". Our criterion would lead us 
to say that sentence (1) is an opinion, because 
"beautiful" is placed in the predicate part and (1) 
is considered to declare the writer’s evaluation 
of the house to a reader. This is why “beautiful” 
in (1) is eligible as a declaratively subjective 
clue. On the other hand, sentence (2) is not 
judged to contain an opinion, because "beauti-
ful" is placed in the noun phrase, i.e., the object 
of the verb “purchase,” and (2) is considered to 
report the event of the house purchase rather ob-
40
jectively to a reader. Sentence (2) partially con-
tains subjective information about the beauty of 
the house; however this information is unlikely 
to be what a writer wants to emphasize. Thus, 
"beautiful" in (2) does not work as a declara-
tively subjective clue. 
These two sentences illustrate the fact that the 
presence of a subjective word (“beautiful”) does 
not unconditionally assure that the sentence ex-
presses an opinion. Additionally, these examples 
do suggest that sentences containing an opinion 
can be judged depending on where such word-
level or phrase-level subjective parts as evalua-
tive adjectives are placed in the predicate part. 
Some word-level or phrase-level subjective 
parts such as subjective sentential adverbs can be 
declaratively subjective clues depending on 
where they occur in the sentence. In sentence (3), 
“amazingly” expresses the writer’s feeling about 
the event. Sentence (3) is judged to contain an 
opinion because there is a subjective sentential 
adverb in its main clause. 
 
(3) Amazingly, few people came to my party. 
 
The existence of some idiomatic collocations 
in the main clause also affects our judgment as 
to what constitutes an opinion-expressing sen-
tence. For example, sentence (4) can be judged 
as expressing an opinion because it includes “my 
wish is”. 
 
(4) My wish is to go abroad. 
 
Thus, depending on the type of declaratively 
subjective clue, it is necessary to consider where 
the expression is placed in the sentence to judge 
whether the sentence is an opinion. 
 
2.2 Clue Expression Collection 
We collected declaratively subjective clues in 
opinion-expressing sentences from Japanese web 
pages. Figure 1 illustrates the flow of collection 
of eligible expressions. 
 
type query’s topic 
Product cell phone, car, beer, cosmetic 
Entertainment sports, movie, game, animation 
Facility  museum, zoo, hotel, shop 
Politics diplomacy, election 
Phenomena diction, social behavior 
Event firework, festival 
Culture artwork, book, music 
Organization company 
Food cuisine, noodle, ice cream 
Creature bird 
Table 1: Topic Examples 
 
First, we retrieved Japanese web pages from 
forty queries covering a wide range of topics 
such as products, entertainment, facilities, and 
phenomena, as shown in Table 1. We used que-
ries on various topics because we wanted to ac-
quire declaratively subjective clues for open-
domain opinion web searches. Most of the que-
ries contain proper nouns. These queries corre-
spond to possible situations in which a user 
wants to retrieve opinions from web pages about 
a particular topic, such as “Cell phone X,” “Y 
museum,” and “Football coach Z’s ability”, 
where X, Y, and Z are proper nouns. 
Next, opinion-expressing sentences were ex-
tracted from the top twenty retrieved web pages 
in each query, 800 pages in total. There were 
75,575 sentences in these pages.  
Figure 1: Flow of Clue Expression Collection 
41
 Three evaluators judged whether each sen-
tence contained an opinion or not. The 13,363 
sentences judged to do so by all three evaluators 
were very likely to be opinion expressing. The 
number of sentences which three evaluators 
agreed on as non-opinion expressing was 
42,346.
1
 Out of the 13,363 opinion expressing 
sentences, 8,425 were then used to extract de-
claratively subjective clues and learn positive 
examples in a Support Vector Machine (SVM), 
and 4,938 were used to assess the performance 
of opinion expressing sentence search (Section 
4). Out of the 42,346 non-opinion sentences, 
26,340 were used to learn negative examples, 
and 16,006 were used to assess, keeping the 
number ratio of the positive and negative exam-
ple sentences in learning and assessing. 
One analyst extracted declaratively subjective 
clues from 8,425 of the 13,363 opinion-
expressing sentences, and another analyst 
checked the result. The number of declaratively 
                                                 
1
 Note that not all of these opinion-expressing sentences 
retrieved were closely related to the query because some of 
the pages described miscellaneous topics.  
subjective clues obtained was 2,936. These clues 
were classified into fourteen types as shown in 
Table 2, where the underlined expressions in 
example sentences are extracted as declaratively 
subjective clues. The example sentences in Table 
2 are Japanese opinion-expressing sentences and 
their English translations. Although some Eng-
lish counterparts of Japanese clue expressions 
might not be cogent because of the characteristic 
difference between Japanese and English, the 
clue types are likely to be language-independent. 
We can see that various types of expressions 
compose opinion-expressing sentences. 
As mentioned in Section 2.1, it is important to 
check where a declaratively subjective clue ap-
pears in the sentence in order to apply our crite-
rion of whether the sentence is an opinion or not. 
The clues in the types other than (b), (c) and (l) 
usually appear in the predicate part of a main 
clause.  
The declaratively subjective clues in Japanese 
examples are placed in the rear parts of sen-
tences except in types (b), (c) and (l). This re-
flects the heuristic rule that Japanese predicate 
 
type 
example sentence 
(English translation of Japanese sentence) 
(a) 
Thought Kono hon wa kare no dato omou. 
(I think this book is his.) 
(b) 
Declarative adverb Tabun rainen yooroppa ni iku. 
(I will possibly go to Europe next year.) 
(c) 
Interjection Waa, suteki. 
(Oh, wonderful.) 
(d) 
Intensifier Karera wa totemo jouzu ni asonda. 
(They played extremely well) 
(e) 
Impression Kono yougo wa yayakoshii. 
(This terminology is confusing.) 
(f) 
Emotion Oai dekite ureshii desu. 
(I am glad to see you.) 
(g) 
Positive/negative judgment Anata no oodio kiki wa sugoi. 
(Your audio system is terrific.) 
(h) 
Modality about propositional attitude Sono eiga wo miru beki da. 
(You should go to the movie.) 
(i) 
Value judgment Kono bun wa imi fumei da. 
(This sentence makes no sense.) 
(j) 
Utterance-specific sentence form Towa ittemo,ima wa tada no yume dakedo. 
(Though, it's literally just a dream now.) 
(k) 
Symbol Keiyaku wo tottazo! 
(We got a contract!)
(l) 
Idiomatic collocation Ii nikui. 
(It's hard to say.) 
(m) 
Uncertainty Ohiru ni nani wo tabeyou kanaa. 
(I am wondering what I should eat for lunch.) 
(n) 
Imperative Saizen wo tukushi nasai. 
(Give it your best.) 
Table 2: Clue Types
42
parts are in principle placed in the rear part of a 
sentence. 
 
3 Opinion-Sentence Extraction 
In this section, we explain the method of classi-
fying each sentence by using declaratively sub-
jective clues. 
The simplest method for automatically judging 
whether a sentence is an opinion is a rule-based 
one that extracts sentences that include declara-
tively subjective clues. However, as mentioned 
in Section 2, the existence of declaratively sub-
jective clues does not assure that the sentence 
expresses an opinion. It is a daunting task to 
write rules that describe how each declaratively 
subjective clue should appear in an opinion-
expressing sentence. A more serious problem is 
that an insufficient collection of declaratively 
subjective clues will lead to poor extraction per-
formance. 
For that reason, we adopted a learning method 
that binarily classifies sentences by using de-
claratively subjective clues and their positions in 
sentences as feature parameters of an SVM. 
With this method, a consistent framework of 
classification can be maintained even if we add 
new declaratively subjective clues, and it is pos-
sible that we can extract the opinion-expressing 
sentences which have unknown declaratively 
subjective clues. 
3.1 Augmentation by Semantic Categories 
Before we can use declaratively subjective clues 
as feature parameters, we must address two is-
sues: 
• Cost of building a corpus:  It is costly 
to provide a sufficient amount of tagged 
corpus of opinion-expressing-sentence la-
bels to ensure that learning achieves a 
high-performance extraction capability. 
• Coverage of words co-occurring with 
declaratively subjective clues:  Many of 
the declaratively subjective clue expres-
sions have co-occurring words in the 
opinion-expressing sentence. Consider the 
following two sentences. 
(5) The sky is high. 
(6) The quality of this product is high. 
 
Both (5) and (6) contain the word "high" 
in the predicate part. Sentence (5) is con-
sidered to be less of an opinion than (6) 
because an evaluator might judge (5) to be 
the objective truth, while all evaluators are 
likely to judge (6) to be an opinion. The 
adjective "high" in the predicate part can 
be validated as a declaratively subjective 
clue depending on co-occurring words. 
However, it is not realistic to provide all 
possible co-occurring words with each 
declaratively subjective clue expression. 
Semantic categories can be of help in dealing 
with the above two issues. Declaratively subjec-
tive clue expressions can be augmented by se-
mantic categories of the words in the expressions. 
An augmentation involving both declaratively 
subjective clues and co-occurrences will increase 
feature parameters. In our implementation, we 
adopted the semantic categories proposed by 
Ikehara et al. (1997). Utilization of semantic 
categories has another effect: it improves the 
extraction performance. Consider the following 
two sentence patterns: 
 
(7) X is beautiful. 
(8) X is pretty. 
 
The words "beautiful" and "pretty" are adjec-
tives in the common semantic category, "appear-
ance", and the degree of declarative subjectivity 
of these sentences is almost the same regardless 
of what X is. Therefore, even if "beautiful" is 
learned as a declaratively subjective clue but 
"pretty" is not, the semantic category "appear-
ance" that the learned word "beautiful" belongs 
to, enables (8) to be judged opinion expressing 
as well as (7). 
3.2 Feature Parameters to Learn 
We implemented our opinion-sentence extrac-
tion method by using a Support Vector Machine 
(SVM) because an SVM can efficiently learn the 
model for classifying sentences into opinion-
expressing and non-opinion expressing, based on 
the combinations of multiple feature parameters. 
The following are the crucial feature parameters 
of our method. 
• 2,936 declaratively subjective clues 
• 2,715 semantic categories that words in 
a sentence can fall into 
If the sentence has a declaratively subjective 
clue of type (b), (c) or (l) in Table 2, the feature 
parameter about the clue is assigned a value of 1; 
if not, it is assigned 0. If the sentence has de-
claratively subjective clues belonging to types 
43
other than (b), (c) or (l) in the predicate part, the 
feature parameter about the clue is assigned 1; if 
not, it is assigned 0. 
The feature parameters for the semantic cate-
gory are used to compensate for the insufficient 
amount of declaratively subjective clues pro-
vided and to consider co-occurring words with 
clue expressions in the opinion-expressing sen-
tences, as mentioned in Section 3.1. 
The following are additional feature parame-
ters. 
• 150 frequent words 
• 13 parts of speech 
Each feature parameter is assigned a value of 1 if 
the sentence has any of the frequent words or 
parts of speech. We added these feature parame-
ters based on the hypotheses that some frequent 
words in Japanese have the function of changing 
the degree of declarative subjectivity, and that 
the existence of such parts of speech as adjec-
tives and adverbs possibly influences the de-
clarative subjectivity. The effectiveness of these 
additional feature parameters was confirmed in 
our preliminary experiment. 
4 Experiments 
We conducted three experiments to assess the 
validity of the proposed method: comparison 
with baseline methods, effectiveness of position 
information in SVM feature parameters, and ef-
fectiveness of SVM feature parameters such as 
declaratively subjective clues and semantic cate-
gories. 
All experiments were performed using the 
Japanese sentences described in Section 2.1.  We 
used 8,425 opinion expressing sentences, which 
were used to collect declaratively subjective 
clues as a training set, and used 4,938 opinion-
expressing sentences as a test set. We also used 
26,340 non-opinion sentences as a training set 
and used 16,006 non-opinion sentences as a test 
set. The test set was divided into ten equal sub-
sets. The experiments were evaluated with the 
following measures following the variable 
scheme in Table 3: 
ba
a
P
op
+
=   
ca
a
R
op
+
=  
opop
opop
op
RP
RP
F
+
=
2
 
dc
d
P
opno
+
=
_
 
db
d
R
opno
+
=
_
 
opnoopno
opnoopno
opno
RP
RP
F
__
__
_
2
+
=  
 
dcba
da
A
+++
+
=  
 
We evaluated ten subsets with the above 
measures and took the average of these results. 
4.1  Comparison with Baseline Methods 
We first performed an experiment comparing 
two baseline methods with our proposed method. 
We prepared a baseline method that regards a 
sentence as an opinion if it contains a number of 
declaratively subjective clues that exceeds a cer-
tain threshold. The best threshold was set 
through trial and error at five occurrences. We 
also prepared another baseline method that 
learns a model and classifies a sentence using 
only features about a bag of words. 
The experimental results are shown in Table 4. 
It can be seen that our method performs better 
than the two baseline methods. Though the dif-
ference between our method’s results and those 
of the bag-of-words method seems rather small, 
the superiority of the proposed method cannot be 
rejected at the significance level of 5% in t-test. 
Answer 
System 
Opinion No opinion 
Opinion a b 
No opinion c d 
Opinion No opinion 
Method 
Precision Recall F-measure Precision Recall F-measure 
Accuracy 
Occurrences of DS clues 
(baseline 1) 
66.4% 35.3% 46.0% 82.6% 94.5% 88.1% 80.5% 
Bag of words 
(baseline 2) 
80.9% 64.2% 71.6% 89.6% 95.3% 92.4% 88.0% 
Proposed 78.6% 70.8% 74.4% 91.3% 94.0% 92.6% 88.6% 
Table 4: Results for comparison with baseline methods 
Table 3: Number of sentences in a test set 
44
 
4.2 Feature Parameters with Position In-
formation 
We inspected the effect of position information 
of 2,936 declaratively subjective clues based on 
the heuristic rule that a Japanese predicate part 
almost always appears in the last ten words in a 
sentence. Instead of more precisely identifying 
predicate position from parsing information, we 
employed this heuristic rule as a feature parame-
ter in the SVM learner for practical reasons. 
Table 5 lists the experimental results. "All 
words" indicates that all feature parameters are 
permitted at any position in the sentence. "Last 
10 words" indicates that all feature parameters 
are permitted only if they occur within the last 
ten words in the sentence.  
We can see that feature parameters with posi-
tion information perform better than those with-
out position information in all evaluations. This 
result confirms our claim that the position of the 
feature parameters is important for judging 
whether a sentence is an opinion or not. 
However, the difference did not indicate supe-
riority between the two results at the significance 
level of 5%. In the “last 10 word” experiment, 
we restricted the position of 422 declaratively 
subjective clues like (b), (c) and (l) in Table 2, 
which appear in any position of a sentence, to 
the same conditions as with the other types of 
2,514 declaratively subjective clues. The fact 
that the equal position restriction on all declara-
tively subjective clues slightly improved per-
formance suggests there will be significant im-
provement in performance from assigning the 
individual position condition to each declara-
tively subjective clue. 
4.3 Effect of Feature Parameters 
The third experiment was designed to ascertain 
the effects of declaratively subjective clues and 
semantic categories. The declaratively subjective 
clues and semantic categories were employed as 
feature parameters for the SVM learner. The ef-
fect of each particular feature parameter can be 
seen by using it without the other feature pa-
rameter, because the feature parameters are in-
dependent of each other. 
The experimental results are shown in Table 6. 
The first row shows trials using only frequent 
words and parts of speech as feature parameters. 
"Y" in the first and second columns indicates 
exclusive use of declaratively subjective clues 
and semantic categories as the feature parame-
ters, respectively. For instance, we can deter-
mine the effect of declaratively subjective clues 
by comparing the first row with the second row. 
The results show the effects of declaratively 
subjective clues and semantic categories. The 
results of the first row show that the method us-
ing only frequent words and parts of speech as 
the feature parameters cannot precisely classify 
subjective sentences. Additionally, the last row 
of the results clearly shows that using both de-
claratively subjective clues and semantic catego-
ries as the feature parameters is the most effec-
tive. The difference between the last row of the 
results and the other rows cannot be rejected 
even at the significance level of 5%. 
Feature sets Opinion No opinion 
DS 
clues 
Semantic 
categories 
Precision Recall F-
measure
Precision Recall F-
measure 
Accuracy 
  71.4% 53.2% 60.9% 87.7% 94.1% 90.8% 85.2% 
Y  79.9% 64.3% 71.2% 89.6% 95.0% 92.2% 87.8% 
 Y 76.1% 68.9% 72.2% 90.7% 93.3% 92.0% 87.5% 
Y Y 78.6% 70.8% 74.4% 91.3% 94.0% 92.6% 88.6% 
Opinion No opinion Position 
Precision Recall F-measure Precision Recall F-measure 
Accuracy 
All words 76.8% 70.6% 73.5% 91.2% 93.4% 92.3% 88.0% 
Last 10 words 78.6% 70.8% 74.4% 91.3% 94.0% 92.6% 88.6% 
Table 5: Results for feature parameters with position information 
Table 6: Results for effect of feature parameters 
45
5 Conclusion and Future Work 
 We proposed a method of extracting sentences 
classified by an SVM as opinion-expressing that 
uses feature sets of declaratively subjective clues 
collected from opinion-expressing sentences in 
Japanese web pages and semantic categories of 
words obtained from a Japanese lexicon. The 
first experiment showed that our method per-
formed better than baseline methods. The second 
experiment suggested that our method performed 
better when extraction of features was limited to 
the predicate part of a sentence rather than al-
lowed anywhere in the sentence. The last ex-
periment showed that using both declaratively 
subjective clues and semantic categories as fea-
ture parameters yielded better results than using 
either clues or categories exclusively. 
Our future work will attempt to develop an 
open-domain opinion web search engine. To 
succeed, we first need to augment the proposed 
opinion-sentence extraction method by incorpo-
rating the query relevancy mechanism. Accord-
ingly, a user will be able to retrieve opinion-
expressing sentences relevant to the query. Sec-
ond, we need to classify extracted sentences in 
terms of emotion, sentiment, requirement, and 
suggestion so that a user can retrieve relevant 
opinions on demand. Finally, we need to sum-
marize the extracted sentences so that the user 
can quickly learn what the writer wanted to say.  

References 
Claire Cardie, Janyce Wiebe, Theresa Wilson, and 
Diane J. Litman. 2003. Combining Low-Level and 
Summary Representations of Opinions. for Multi-
Perspective Question Answering. Working Notes - 
New Directions in Question Answering (AAAI 
Spring Symposium Series) . 
Kushal Dave, Steve Lawrence, and David M. Pennock. 
2003. Mining the peanut gallery: Opinion extraction 
and semantic classification of product reviews. Pro-
ceedings of the 12th International World Wide Web 
Conference, 519-528. 
Satoru Ikehara, Masahiro Miyazaki, Akio Yokoo, Sato-
shi Shirai, Hiromi Nakaiwa, Kentaro Ogura, Yoshi-
fumi Ooyama, and Yoshihiko Hayashi. 1997. Ni-
hongo Goi Taikei – A Japanese Lexicon. Iwanami 
Shoten. 5 volumes. (In Japanese). 
Soo-Min Kim and Eduard Hovy. 2004. Determining the 
Sentiment of Opinions. Proceedings of the. COLING-
04. 
Nozomi Kobayashi, Kentaro Inui, Yuji Matsumoto, 
Kenji Tateishi, and Toshikazu Fukushima. 2004. Col-
lecting Evaluative Expressions for Opinion Extrac-
tion. Proceedings of the First International Joint Con-
ference on Natural Language Processing (IJCNLP-
04), 584-589. 
Satoshi Morinaga, Kenji Yamanishi, and Kenji Tateishi. 
2002. Mining Product Reputations on the Web. Pro-
ceedings of the eighth ACM SIGKDD International 
Conference on Knowledge Discovery and Data Min-
ing (KDD 2002).  
Bo Pang, Lillian Lee, and Shivakumar Vaithyanathan. 
2002. Thumbs up? Sentiment Classification using 
Machine Learning Techniques. Proceedings of the 
Conference on Empirical Methods in Natural Lan-
guage Processing (EMNLP-2002), 76-86. 
Tetsuya Nasukawa and Jeonghee Yi. 2003. Sentiment 
Analysis: Capturing Favorability Using Natural 
Language Processing. Proceedings of the 2nd Inter-
national Conference on Knowledge Capture(K-CAP 
2003). 
Ellen Riloff and Janyce Wiebe. 2003. Learning Extrac-
tion Patterns for Subjective Expressions. Proceedings 
of the Conference on Empirical Methods in Natural 
Language Processing (EMNLP-03), 105-112. 
Kenji Tateishi, Yoshihide Ishiguro, and Toshikazu Fu-
kushima, 2004. A Reputation Search Engine that 
Collects People’s Opinions by Information Extrac-
tion Technology, IPSJ Transactions Vol. 45 
No.SIG07, 115-123. 
Peter Turney. 2002. Thumbs Up or Thumbs Down? Se-
mantic Orientation Applied to Unsupervised Classifi-
cation of Reviews. Proceedings of the 40th Annual 
Meeting of the Association for Computational Lin-
guistics (ACL-2002), 417-424. 
Janyce Wiebe. 2000. Learning Subjective Adjectives 
from Corpora. Proceedings of the 17th National Con-
ference on Artificial Intelligence (AAAI -2000).  
Janyce Wiebe, Theresa Wilson, and Matthew Bell. 2001. 
Identifying Collocations for Recognizing Opinions. 
Proceedings of ACL/EACL 2001 Workshop on Col-
location.  
Janyce Wiebe and Ellen Riloff. 2005. Creating Subjec-
tive and Objective Sentence Classifiers from Unanno-
tated Texts. Proceedings of Sixth International Con-
ference on Intelligent Text Processing and Computa-
tional Linguistics (CICLing-2005), 486-497.  
Theresa Wilson, Janyce Wiebe, and Rebecca Hwa, 2004. 
Just how mad are you? Finding strong and weak 
opinion clauses. Proceeding of the AAAI Spring 
Symposium on Exploring Attitude and Affect in 
Text: Theories and Applications. 
Hong Yu and Vasileios Hatzivassiloglou. 2003. To-
wards Answering Opinion Questions: Separating 
Facts from Opinions and Identifying the Polarity of 
Opinion Sentences. Proceedings of the Conference 
on Empirical Methods in Natural Language Process-
ing (EMNLP-2003). 
