<?xml version="1.0" standalone="yes"?>
<Paper uid="W05-0408">
  <Title>tion with known sentiment terms</Title>
  <Section position="7" start_page="62" end_page="63" type="concl">
    <SectionTitle>
5 Discussion
</SectionTitle>
    <Paragraph position="0"/>
    <Section position="1" start_page="62" end_page="62" type="sub_section">
      <SectionTitle>
5.1 A note on statistical significance
</SectionTitle>
      <Paragraph position="0"> We used the McNemar test to assess whether two classifiers are performing significantly differently.</Paragraph>
      <Paragraph position="1"> This test establishes whether the accuracy of two classifiers differs significantly - it does not guarantee significance for precision and recall differences. For the latter, other tests have been proposed (e.g. Chinchor 1995), but time constraints prohibited us from implementing any of those more computationally costly tests.</Paragraph>
      <Paragraph position="2"> For the results presented in the previous sections the McNemar test established statistical significance at the 0.99 level over baseline (i.e. the SO results in Table 1) for the multiple iterations results (Table 4) and the bootstrapping approach (Table 5), but not for the SM+SO approach (Table 2).</Paragraph>
    </Section>
    <Section position="2" start_page="62" end_page="63" type="sub_section">
      <SectionTitle>
5.2 Future work
</SectionTitle>
      <Paragraph position="0"> This exploratory set of experiments indicates a number of interesting directions for future work. A shortcoming of the present work is the manual tuning of cutoff parameters. This problem could be alleviated in at least two possible ways: First, using a general combination of the ranking of terms according to SM and SO. In other words, calculate the semantic weight of a term as a combination of SO and its rank in the SM scores.</Paragraph>
      <Paragraph position="1">  Secondly, following a suggestion by an anonymous reviewer, the Naive Bayes bootstrapping approach could be used in a feedback loop to inform the SO score estimation in the absence of a manually annotated parameter tuning set.</Paragraph>
    </Section>
    <Section position="3" start_page="63" end_page="63" type="sub_section">
      <SectionTitle>
5.3 Summary
</SectionTitle>
      <Paragraph position="0"> Our results demonstrate that the SM method can serve as a valid tool to mine sentiment-rich vocabulary in a domain. SM will yield a list of terms that are likely to have a strong sentiment orientation. SO can then be used to find the polarity for the selected features by association with the sentiment terms of known polarity in the seed word list.</Paragraph>
      <Paragraph position="1"> Performing this process iteratively by first enhancing the set of seed words through SM+SO yields the best results. While this approach does not compare to the results that can be achieved by supervised learning with large amounts of labeled data, it does improve on results obtained by using SO alone.</Paragraph>
      <Paragraph position="2"> We believe that this result is relevant in two respects. First, by improving average precision and recall on the classification task, we move closer to the goal of unsupervised sentiment classification.</Paragraph>
      <Paragraph position="3"> This is a very important goal in itself given the need for &amp;quot;out of the box&amp;quot; sentiment techniques in business intelligence and the notorious difficulty of rapidly adapting to a new domain (Engstrom 2004, Aue and Gamon 2005). Second, the exploratory results reported here may indicate a general source of information for feature selection in natural language tasks: features that have a tendency to be in complementary distribution (especially in smaller linguistic units such as sentences) may often form a class that shares certain properties. In other words, it is not only the strong association scores that should be exploited but also the particularly weak (negative) associations.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>