Proceedings of HLT/EMNLP 2005 Demonstration Abstracts, pages 32–33,
Vancouver, October 2005.
OPINE: Extracting Product Features and Opinions from Reviews
Ana-Maria Popescu Bao Nguyen
Department of Computer Science and Engineering
University of Washington
Seattle, WA 98195-2350
{amp,omicron,etzioni}@cs.washington.edu
Oren Etzioni
Abstract
Consumers have to often wade
through a large number of on-line re-
views in order to make an informed
product choice. We introduce OPINE,
an unsupervised, high-precision in-
formation extraction system which
mines product reviews in order to
build a model of product features and
their evaluation by reviewers.
1 Introduction
The Web contains a wealth of customer reviews - as a
result, the problem of “review mining” has seen increas-
ing attention over the last few years from (Turney, 2003;
Hu and Liu, 2004) and many others. We decompose the
problem of review mining into the following subtasks:
a) Identify product features, b) Identify opinions re-
garding product features, c) Determine the polarity of
each opinion and d) Rank opinions according to their
strength (e.g., “abominable” is stronger than “bad”).
We introduce OPINE, an unsupervised information ex-
traction system that embodies a solution to each of the
above subtasks. The remainder of this paper is organized
as follows: Section 2 describes OPINE’s components to-
gether with their experimental evaluation and Section 3
describes the related work.
2 OPINE Overview
OPINE is built on top of KNOWITALL, a Web-based,
domain-independent information extraction system (Et-
zioni et al., 2005). Given a set of relations of inter-
est, KNOWITALL instantiates relation-specific generic
extraction patterns into extraction rules which find can-
didate facts. The Assessor module then assigns a proba-
bility to each candidate using a form of Point-wise Mu-
tual Information (PMI) between phrases that is estimated
from Web search engine hit counts (Turney, 2003). It
Input: product class C, reviews R.
Output: set of [feature, ranked opinion list] tuples
R’ parseReviews(R);
E findExplicitFeatures(R’, C);
O findOpinions(R’, E);
CO clusterOpinions(O);
I findImplicitFeatures(CO, E);
RO rankOpinions(CO);
f(f, oi, ...oj)g outputTuples(RO, I[E);
Figure 1: OPINE Overview.
computes the PMI between each fact and discriminator
phrases (e.g., “is a scanner” for the isA() relationship
in the context of the Scanner class). Given fact f and
discriminator d, the computed PMI score is:
PMI(f, d) = Hits(d + f)Hits(d)∗Hits(f)
The PMI scores are converted to binary features for a
Naive Bayes Classifier, which outputs a probability asso-
ciated with each fact.
Given product class C with instances I and reviews R,
OPINE’s goal is to find the set of (feature, opinions) tuples
f(f, oi, ...oj)gs.t. f 2F and oi, ...oj 2O, where:
a) F is the set of product class features in R.
b) O is the set of opinion phrases in R.
c) opinions associated with a particular feature are
ranked based on their strength.
OPINE’s solution to this task is outlined in Figure 1. In
the following, we describe in detail each step.
Explicit Feature Extraction OPINE parses the re-
views using the MINIPAR dependency parser (Lin, 1998)
and applies a simple pronoun-resolution module to the
parsed data. The system then finds explicitly men-
tioned product features (E) using an extended version
of KNOWITALL’s extract-and-assess strategy described
above. OPINE extracts the following types of product fea-
tures: properties, parts, features of product parts (e.g.,
ScannerCoverSize), related concepts (e.g., Image
32
is related to Scanner) and parts and properties of re-
lated concepts (e.g., ImageSize). When compared on
this task with the most relevant previous review-mining
system in (Hu and Liu, 2004), OPINE obtains a 22% im-
provement in precision with only a 3% reduction in recall
on the relevant 5 datasets. One third of this increase is due
to OPINE’s feature assessment step and the rest is due to
the use of Web PMI statistics.
Opinion Phrases OPINE extracts adjective, noun, verb
and adverb phrases attached to explicit features as poten-
tial opinion phrases. OPINE then collectively assigns pos-
itive, negative or neutral semantic orientation (SO) labels
to their respective head words. This problem is similar to
labeling problems in computer vision and OPINE uses a
well-known computer vision technique, relaxation label-
ing, as the basis of a 3-step SO label assignment proce-
dure. First, OPINE identifies the average SO label for a
word w in the context of the review set. Second, OPINE
identifies the average SO label for each word w in the
context of a feature f and of the review set (“hot” has
a negative connotation in “hot room”, but a positive one
in “hot water”). Finally, OPINE identifies the SO label of
word w in the context of feature f and sentence s. For ex-
ample, some people like large scanners (“I love this large
scanner”) and some do not (“I hate this large scanner”).
The phrases with non-neutral head words are retained as
opinion phrases and their polarity is established accord-
ingly. On the task of opinion phrase extraction, OPINE
obtains a precision of 79% and a recall of 76% and on the
task of opinion phrase polarity extraction OPINE obtains
a precision of 86% and a recall of 84%.
Implicit Features Opinion phrases refer to properties,
which are sometimes implicit (e.g., “tiny phone” refers to
the phone size). In order to extract such properties, OPINE
first clusters opinion phrases (e.g., tiny and small will
be placed in the same cluster), automatically labels the
clusters with property names (e.g., Size) and uses them
to build implicit features (e.g., PhoneSize). Opinion
phrases are clustered using a mixture of WordNet infor-
mation (e.g., antonyms are placed in the same cluster) and
lexical pattern information (e.g., “clean, almost spotless”
suggests that “clean” and “spotless” are likely to refer to
the same property). (Hu and Liu, 2004) doesn’t handle
implicit features, so we have evaluated the impact of im-
plicit feature extraction on two separate sets of reviews
in the Hotels and Scanners domains. Extracting implicit
features (in addition to explicit features) has resulted in a
2% increase in precision and a 6% increase in recall for
OPINE on the task of feature extraction.
Ranking Opinion Phrases Given an opinion cluster,
OPINE uses the final probabilities associated with the SO
labels in order to derive an initial opinion phrase strength
ranking (e.g., great > good > average) in the manner
of (Turney, 2003). OPINE then uses Web-derived con-
straints on the relative strength of phrases in order to im-
prove this ranking. Patterns such as “a1, (*) even a2” are
good indicators of how strong a1 is relative to a2. OPINE
bootstraps a set of such patterns and instantiates them
with pairs of opinions in order to derive constraints such
as strength(deafening) > strength(loud). OPINE
also uses synonymy and antonymy-based constraints
such as strength(clean) = strength(dirty). The con-
straint set induces a constraint satisfaction problem
whose solution is a ranking of the respective cluster opin-
ions (the remaining opinions maintain their default rank-
ing). OPINE’s accuracy on the opinion ranking task is
87%. Finally, OPINE outputs a set of (feature, ranked
opinions) tuples for each product.
3 Related Work
The previous review-mining systems most relevant to
our work are (Hu and Liu, 2004) and (Kobayashi et
al., 2004). The former’s precision on the explicit fea-
ture extraction task is 22% lower than OPINE’s while
the latter employs an iterative semi-automatic approach
which requires significant human input; neither handles
implicit features. Unlike previous research on identifying
the subjective character and the polarity of phrases and
sentences ((Hatzivassiloglou and Wiebe, 2000; Turney,
2003) and many others), OPINE identifies the context-
sensitive polarity of opinion phrases. In contrast to super-
vised methods which distinguish among strength levels
for sentences or clauses ((Wilson et al., 2004) and oth-
ers), OPINEuses an unsupervised constraint-based opin-
ion ranking approach.
References
O. Etzioni, M. Cafarella, D. Downey, S. Kok, A. Popescu,
T. Shaked, S. Soderland, D. Weld, and A. Yates. 2005. Un-
supervised named-entity extraction from the web: An exper-
imental study. Artificial Intelligence, 165(1):91–134.
V. Hatzivassiloglou and J. Wiebe. 2000. Effects of Adjec-
tive Orientation and Gradability on Sentence Subjectivity. In
COLING, pages 299–305.
M. Hu and B. Liu. 2004. Mining and Summarizing Customer
Reviews. In KDD, pages 168–177, Seattle, WA.
N. Kobayashi, K. Inui, K. Tateishi, and T. Fukushima. 2004.
Collecting Evaluative Expressions for Opinion Extraction.
In IJCNLP, pages 596–605.
D. Lin. 1998. Dependency-based evaluation of MINIPAR. In
Workshop on Evaluation of Parsing Systems at ICLRE.
P. Turney. 2003. Inference of Semantic Orientation from Asso-
ciation. In CoRR cs. CL/0309034.
T. Wilson, J. Wiebe, and R. Hwa. 2004. Just how mad are you?
finding strong and weak opinion clauses. In AAAI, pages
761–769.
33
