<?xml version="1.0" standalone="yes"?>
<Paper uid="N06-1026">
  <Title>Identifying and Analyzing Judgment Opinions</Title>
  <Section position="2" start_page="0" end_page="200" type="intro">
    <SectionTitle>
1 Introduction
</SectionTitle>
    <Paragraph position="0"> Recently, many researchers and companies have explored the area of opinion detection and analysis.</Paragraph>
    <Paragraph position="1"> With the increased immersion of Internet users has come a proliferation of opinions available on the web. Not only do we read more opinions from the web, such as in daily news editorials, but also we post more opinions through mechanisms such as governmental web sites, product review sites, news group message boards and personal blogs. This phenomenon has opened the door for massive opinion collection, which has potential impact on various applications such as public opinion monitoring and product review summary systems.</Paragraph>
    <Paragraph position="2"> Although in its infancy, many researchers have worked in various facets of opinion analysis. Pang et al. (2002) and Turney (2002) classified sentiment polarity of reviews at the document level.</Paragraph>
    <Paragraph position="3"> Wiebe et al. (1999) classified sentence level subjectivity using syntactic classes such as adjectives, pronouns and modal verbs as features. Riloff and Wiebe (2003) extracted subjective expressions from sentences using a bootstrapping pattern learning process. Yu and Hatzivassiloglou (2003) identified the polarity of opinion sentences using semantically oriented words. These techniques were applied and examined in different domains, such as customer reviews (Hu and Liu 2004) and news articles  . These researchers use lists of opinion-bearing clue words and phrases, and then apply various additional techniques and refinements.</Paragraph>
    <Paragraph position="4"> Along with many opinion researchers, we participated in a large pilot study, sponsored by NIST, which concluded that it is very difficult to define what an opinion is in general. Moreover, an expression that is considered as an opinion in one domain might not be an opinion in another. For example, the statement &amp;quot;The screen is very big&amp;quot; might be a positive review for a wide screen desk-top review, but it could be a mere fact in general newspaper text. This implies that it is hard to apply opinion bearing words collected from one domain to an application for another domain. One might therefore need to collect opinion clues within individual domains. In case we cannot simply find training data from existing sources, such as news article analysis, we need to manually annotate data first.</Paragraph>
    <Paragraph position="5"> Most opinions are of two kinds: 1) beliefs about the world, with values such as true, false, possible, unlikely, etc.; and 2) judgments about the world, with values such as good, bad, neutral, wise, foolish, virtuous, etc. Statements like &amp;quot;I believe that he is smart&amp;quot; and &amp;quot;Stock prices will rise soon&amp;quot; are examples of beliefs whereas &amp;quot;I like the new policy on social security&amp;quot; and &amp;quot;Unfortunately this really was his year: despite a stagnant economy, he still won his re-election&amp;quot; are examples of judgment opinions. However, judgment opinions and beliefs are not necessarily mutually exclusive. For example, &amp;quot;I think it is an outrage&amp;quot; or &amp;quot;I believe that he is smart&amp;quot; carry both a belief and a judgment.</Paragraph>
    <Paragraph position="6"> In the NIST pilot study, it was apparent that human annotators often disagreed on whether a belief statement was or was not an opinion. However, high annotator agreement was seen on judg- null ment opinions. In this paper, we therefore focus our analysis on judgment opinions only. We hope that future work yields a more precise definition of belief opinions on which human annotators can agree.</Paragraph>
    <Paragraph position="7"> We define a judgment opinion as consisting of three elements: a valence, a holder, and a topic.</Paragraph>
    <Paragraph position="8"> The valence, which applies specifically to judgment opinions and not beliefs, is the value of the judgment. In our framework, we consider the following valences: positive, negative, and neutral.</Paragraph>
    <Paragraph position="9"> The holder of an opinion is the person, organization or group whose opinion is expressed. Finally, the topic is the event or entity about which the opinion is held.</Paragraph>
    <Paragraph position="10"> In previous work, Choi et al. (2005) identify opinion holders (sources) using Conditional Random Fields (CRF) and extraction patterns. They define the opinion holder identification problem as a sequence tagging task: given a sequence of words</Paragraph>
    <Paragraph position="12"> ) in a sentence, they generate a sequence of labels (</Paragraph>
    <Paragraph position="14"> ) indicating whether the word is a holder or not. However, there are many cases where multiple opinions are expressed in a sentence each with its own holder. In those cases, finding opinion holders for each individual expression is necessary. In the corpus they used, 48.5% of the sentences which contain an opinion have more than one opinion expression with multiple opinion holders. This implies that multiple opinion expressions in a sentence occur significantly often. A major challenge of our work is therefore not only to focus on sentence with only one opinion, but also to identify opinion holders when there is more than one opinion expressed in a sentence. For example, consider the sentence &amp;quot;In relation to Bush's axis of evil remarks, the German Foreign Minister also said, Allies are not satellites, and the French Foreign Minister caustically criticized that the United States' unilateral, simplistic worldview poses a new threat to the world&amp;quot;. Here, &amp;quot;the German Foreign Minister&amp;quot; should be the holder for the opinion &amp;quot;Allies are not satellites&amp;quot; and &amp;quot;the French Foreign Minister&amp;quot; should be the holder for &amp;quot;caustically criticized&amp;quot;.</Paragraph>
    <Paragraph position="15"> In this paper, we introduce a methodology for analyzing judgment opinions. We decompose the task into four parts: 1) recognizing the opinion; 2) identifying the valence; 3) identifying the holder; and 4) identifying the topic. For the purposes of this paper, we address the first three parts and leave the last for future work. Opinions can be extracted from various granularities such as a word, a sentence, a text, or even multiple texts. Each is important, but we focus our attention on word-level opinion detection (Section 2.1) and the detection of opinions in short emails (Section 3). We evaluate our methodology using intrinsic and extrinsic measures.</Paragraph>
    <Paragraph position="16"> The remainder of the paper is organized as follows. In the next section, we describe our methodology addressing the three steps described above, and in Section 4 we present our experimental results. We conclude with a discussion of future work.</Paragraph>
  </Section>
class="xml-element"></Paper>