<?xml version="1.0" standalone="yes"?>
<Paper uid="N06-3005">
  <Title>Identifying Perspectives at the Document and Sentence Levels Using Statistical Models</Title>
  <Section position="7" start_page="228" end_page="229" type="evalu">
    <SectionTitle>
5 Experiments
</SectionTitle>
    <Paragraph position="0"/>
    <Section position="1" start_page="228" end_page="228" type="sub_section">
      <SectionTitle>
5.1 Identifying Perspectives at the Document
Level
</SectionTitle>
      <Paragraph position="0"> To objectively evaluate how well na&amp;quot;ive Bayes models (NB) learn to identify perspectives expressed at the document level, we train NB against on the bitterlemons corpus, and evaluate how accurately NB predicts the perspective of a unseen document as either Palestinian or Israeli in ten-fold cross-validation manner. The average classification accuracy over 10 folds is reported. We compare three different models, including NB with two different inference methods and Support Vector Ma-</Paragraph>
    </Section>
    <Section position="2" start_page="228" end_page="228" type="sub_section">
      <SectionTitle>
Document Level
</SectionTitle>
      <Paragraph position="0"> The results in Table 1 show that both NB and SVM perform surprisingly well on both Editors and Guests subsets of the bitterlemons corpus. We also see that NBs further reduce classification errors even though SVM already achieves high accuracy. By considering the full posterior distribution NB-B further improves on NB-M, which performs only point estimation. The results strongly suggest that the word choices made by authors, either consciously or subconsciously, reflect much of their political perspectives.</Paragraph>
    </Section>
    <Section position="3" start_page="228" end_page="229" type="sub_section">
      <SectionTitle>
5.2 Identifying Perspectives at the Sentence
Level
</SectionTitle>
      <Paragraph position="0"> In addition to identify the perspectives of a document, we are interested in which sentences in the document strongly convey perspectives. Although the posterior probability that a sentence  covey strongly perspectives in (6) is of our interest, we can not directly evaluate their quality due to the lack of golden truth at the sentence level. Alternatively we evaluate how accurately LSPM predicts the perspective of a document, in the same way of evaluating SVM and NB in the previous section. If LSPM does not achieve similar identification accuracy after modeling sentence-level information, we will doubt the quality of predictions on how strongly a sentence convey perspective made by LSPM.</Paragraph>
    </Section>
    <Section position="4" start_page="229" end_page="229" type="sub_section">
      <SectionTitle>
Sentence Level
</SectionTitle>
      <Paragraph position="0"> The experimental results in Table 2 show that the LSPM achieves similarly or even slightly better accuracy than those of NBs, which is very encouraging and suggests that the proposed LSPM closely match how perspectives are expressed at the document and sentence levels. If one does not explicitly model the uncertainty at the sentence level, one can train NB directly against the sentences to classify a sentence into Palestinian or Israeli perspective. We obtain the accuracy of 0.7529, which is much lower than the accuracy previously achieved at the document level. Therefore identifying perspective at the sentence level is much harder than at that the document level, and the high accuracy of identifying document-level perspectives suggests that LPSM closely captures the perspectives expressed at the document and sentence levels, given individual sentences are very short and much less informative about overall perspective.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>