<?xml version="1.0" standalone="yes"?>
<Paper uid="P04-3025">
  <Title>Incorporating topic information into sentiment analysis models</Title>
  <Section position="5" start_page="0" end_page="0" type="metho">
    <SectionTitle>
3 Methods
</SectionTitle>
    <Paragraph position="0"/>
    <Section position="1" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
3.1 Semantic orientation with PMI
</SectionTitle>
      <Paragraph position="0"> Here, the term semantic orientation (SO) (Hatzivassiloglou and McKeown, 2002) refers to a real number measure of the positive or negative sentiment expressed by a word or phrase. In the present work, the approach taken by Turney (2002) is used to derive such values for selected phrases in the text. For the purposes of this paper, these phrases will be referred to as value phrases, since they will be the sources of SO values. Once the desired value phrases have been extracted from the text, each one is assigned an SO value. The SO of a phrase is determined based upon the phrase's pointwise mutual information (PMI) with the words &amp;quot;excellent&amp;quot; and &amp;quot;poor&amp;quot;. PMI is defined by Church and Hanks (1989) as follows:</Paragraph>
      <Paragraph position="2"> where a28 a5a8a7a10a9a19a30a31a7a15a14a12a16 is the probability that a7a34a9 and a7a35a14 co-occur.</Paragraph>
      <Paragraph position="3"> The SO for a a28a37a36a39a38a41a40a29a42a44a43 is the difference between its PMI with the word &amp;quot;excellent&amp;quot; and its PMI with the word &amp;quot;poor.&amp;quot; The method used to derive these values takes advantage of the possibility of using the World Wide Web as a corpus, similarly to work such as (Keller and Lapata, 2003). The probabilities are estimated by querying the AltaVista Advanced Search engine1 for counts. The search engine's &amp;quot;NEAR&amp;quot; operator, representing occurrences of the two queried words within ten words of each other in a text, is used to define co-occurrence. The final SO equation is</Paragraph>
      <Paragraph position="5"> Intuitively, this yields values above zero for phrases with greater PMI with the word &amp;quot;excellent&amp;quot; and below zero for greater PMI with &amp;quot;poor&amp;quot;. A SO value of zero would indicate a completely neutral semantic orientation.</Paragraph>
    </Section>
    <Section position="2" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
3.2 Osgood semantic differentiation with WordNet
</SectionTitle>
      <Paragraph position="0"> Further feature types are derived using the method of Kamps and Marx (2002) of using WordNet relationships to derive three values pertinent to the emotive meaning of adjectives. The three values correspond to the potency (strong or weak), activity (active or passive) and the evaluative (good or bad) factors introduced in Charles Osgood's Theory of Semantic Differentiation (Osgood et al., 1957).</Paragraph>
      <Paragraph position="1"> These values are derived by measuring the relative minimal path length (MPL) in WordNet between the adjective in question and the pair of words appropriate for the given factor. In the case of the evaluative factor (EVA) for example, the comparison is between the MPL between the adjective and &amp;quot;good&amp;quot; and the MPL between the adjective and &amp;quot;bad&amp;quot;.</Paragraph>
      <Paragraph position="2"> Only adjectives connected by synonymy to each of the opposites are considered. The method results in a list of 5410 adjectives, each of which is given a value for each of the three factors referred to as EVA, POT, and ACT. Each of these factors' values are averaged over all the adjectives in a text, yielding three real-valued feature values for the text, which will be added to the SVM model.</Paragraph>
    </Section>
    <Section position="3" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
3.3 Topic proximity and syntactic-relation features
</SectionTitle>
      <Paragraph position="0"> In some application domains, it is known in advance what the topic is toward which sentiment is to be evaluated. Incorporating this information is done by creating several classes of features based upon the semantic orientation values of phrases given their position in relation to the topic of the text. The approach allows secondary information to be incorporated where available, in this case, the primary information is the specific record being reviewed and the secondary information identified is the artist.</Paragraph>
      <Paragraph position="1"> Texts were annotated by hand using the Open Ontology Forge annotation tool (Collier et al., 2003).</Paragraph>
      <Paragraph position="2"> In each record review, references (including co-reference) to the record being reviewed were tagged as THIS WORK and references to the artist under review were tagged as THIS ARTIST.</Paragraph>
      <Paragraph position="3"> With these entities tagged, a number of classes of features may be extracted, representing various relationships between topic entities and value phrases similar to those described in section 3.1. The classes looked at in this work are as follows: Turney Value The average value of all value phrases' SO values for the text. Classification by this feature alone is not the equivalent of Turney's approach, since the present approach involves retraining in a supervised model.</Paragraph>
      <Paragraph position="4"> In sentence with THIS WORK The average value of all value phrases which occur in the same sentence as a reference to the work being reviewed.</Paragraph>
      <Paragraph position="5">  being reviewed directly, or separated only by the copula or a preposition.</Paragraph>
      <Paragraph position="6"> In sentence with THIS ARTIST As above, but with reference to the artist.</Paragraph>
      <Paragraph position="7"> Following THIS ARTIST As above, but with reference to the artist.</Paragraph>
      <Paragraph position="8"> Preceding THIS ARTIST As above, but with reference to the artist.</Paragraph>
      <Paragraph position="9"> The features used which make use of adjectives with WordNet derived Osgood values include the following: null Text-wide EVA The average EVA value of all adjectives in a text.</Paragraph>
      <Paragraph position="10"> Text-wide POT The average POT value of all adjectives in a text.</Paragraph>
      <Paragraph position="11"> Text-wide ACT The average ACT value of all adjectives in a text.</Paragraph>
      <Paragraph position="12"> TOPIC-sentence EVA The average EVA value of all adjectives which share a sentence with the topic of the text.</Paragraph>
      <Paragraph position="13"> TOPIC-sentence POT The average POT value of all adjectives which share a sentence with the topic of the text.</Paragraph>
      <Paragraph position="14"> TOPIC-sentence ACT The average ACT value of all adjectives which share a sentence with the topic of the text.</Paragraph>
      <Paragraph position="15"> The grouping of these classes should reflect some common degree of reliability of features within a given class, but due to data sparseness what might have been more natural class groupings--for example including value-phrase preposition topic-entity as a distinct class--often had to be conflated in order to get features with enough occurrences to be representative.</Paragraph>
    </Section>
  </Section>
  <Section position="6" start_page="0" end_page="0" type="metho">
    <SectionTitle>
4 Experiments
</SectionTitle>
    <Paragraph position="0"> The dataset consists of 100 record reviews from the Pitchfork Media online record review publication,2 topic-annotated by hand. Features used include word unigrams and lemmatized unigrams3 as well as the features described in 3.3 which make use of topic information, namely the broader PMI derived SO values and the topic-sentence Osgood values. Due to the relatively small size of this dataset, test suites were created using 100, 20, 10, and 5-fold cross validation, to maximize the amount of data available for training and the accuracy of the results.</Paragraph>
    <Paragraph position="1"> SVMs were built using Kudo's TinySVM software implementation.4</Paragraph>
  </Section>
class="xml-element"></Paper>