<?xml version="1.0" standalone="yes"?>
<Paper uid="J96-2004">
  <Title>Squibs and Discussions Assessing Agreement on Classification Tasks: The Kappa Statistic</Title>
  <Section position="6" start_page="252" end_page="252" type="concl">
    <SectionTitle>
5. Conclusions
</SectionTitle>
    <Paragraph position="0"> We have shown that existing measures of reliability in discourse and dialogue work are difficult to interpret, and we have suggested a replacement measure, the kappa statistic, which has a number of advantages over these measures. Kappa is widely accepted in the field of content analysis. It is interpretable, allows different results to be compared, and suggests a set of diagnostics in cases where the reliability results are not good enough for the required purpose. We suggest that this measure be adopted more widely within our own research community.</Paragraph>
  </Section>
class="xml-element"></Paper>