<?xml version="1.0" standalone="yes"?>
<Paper uid="P05-1015">
  <Title>respect to rating scales</Title>
  <Section position="3" start_page="0" end_page="115" type="intro">
    <SectionTitle>
1 Introduction
</SectionTitle>
    <Paragraph position="0"> There has recently been a dramatic surge of interest in sentiment analysis, as more and more people become aware of the scienti c challenges posed and the scope of new applications enabled by the processing of subjective language. (The papers collected by Qu, Shanahan, and Wiebe (2004) form a representative sample of research in the area.) Most prior work on the speci c problem of categorizing expressly opinionated text has focused on the binary distinction of positive vs. negative (Turney, 2002; Pang, Lee, and Vaithyanathan, 2002; Dave, Lawrence, and Pennock, 2003; Yu and Hatzivassiloglou, 2003). But it is often helpful to have more information than this binary distinction provides, especially if one is ranking items by recommendation or comparing several reviewers' opinions: example applications include collaborative ltering and deciding which conference submissions to accept.</Paragraph>
    <Paragraph position="1"> Therefore, in this paper we consider generalizing to ner-grained scales: rather than just determine whether a review is thumbs up or not, we attempt to infer the author's implied numerical rating, such as three stars or four stars . Note that this differs from identifying opinion strength (Wilson, Wiebe, and Hwa, 2004): rants and raves have the same strength but represent opposite evaluations, and referee forms often allow one to indicate that one is very con dent (high strength) that a conference submission is mediocre (middling rating). Also, our task differs from ranking not only because one can be given a single item to classify (as opposed to a set of items to be ordered relative to one another), but because there are settings in which classi cation is harder than ranking, and vice versa.</Paragraph>
    <Paragraph position="2"> One can apply standarda6 -ary classi ers or regression to this rating-inference problem; independent work by Koppel and Schler (2005) considers such  methods. But an alternative approach that explicitly incorporates information about item similarities together with label similarity information (for instance, one star is closer to two stars than to four stars ) is to think of the task as one of metric labeling (Kleinberg and Tardos, 2002), where label relations are encoded via a distance metric.</Paragraph>
    <Paragraph position="3"> This observation yields a meta-algorithm, applicable to both semi-supervised (via graph-theoretic techniques) and supervised settings, that alters a given a6 -ary classi er's output so that similar items tend to be assigned similar labels.</Paragraph>
    <Paragraph position="4"> In what follows, we rst demonstrate that humans can discern relatively small differences in (hidden) evaluation scores, indicating that rating inference is indeed a meaningful task. We then present three types of algorithms one-vs-all, regression, and metric labeling that can be distinguished by how explicitly they attempt to leverage similarity between items and between labels. Next, we consider what item similarity measure to apply, proposing one based on the positive-sentence percentage. Incorporating this new measure within the metric-labeling framework is shown to often provide signi cant improvements over the other algorithms.</Paragraph>
    <Paragraph position="5"> We hope that some of the insights derived here might apply to other scales for text classifcation that have been considered, such as clause-level opinion strength (Wilson, Wiebe, and Hwa, 2004); affect types like disgust (Subasic and Huettner, 2001; Liu, Lieberman, and Selker, 2003); reading level (Collins-Thompson and Callan, 2004); and urgency or criticality (Horvitz, Jacobs, and Hovel, 1999).</Paragraph>
  </Section>
class="xml-element"></Paper>