<?xml version="1.0" standalone="yes"?>
<Paper uid="W04-3217">
  <Title>Automatic Analysis of Plot for Story Rewriting</Title>
  <Section position="9" start_page="0" end_page="0" type="evalu">
    <SectionTitle>
8 Discussion
</SectionTitle>
    <Paragraph position="0"> From these experiments as shown in Table 6 we see that the type of machine learner and the particular features are important to correctly classify children's stories. Inspection of the results shows that separating good and excellent stories from poor stories is best performed by Naive Bayes. For our application, teachers have indicated that the classification of an excellent or good story as a poor one is considered worse than the classifying of a fair or even poor story as good. Moreover, it uses the event-based results of the Plot Comparison Algorithm so that the agent in StoryStation may use these results to inform the student what precise events and entities are missing or misused. NB is fast enough to provide possible feedback in real time and its ability to separate poor stories from good and excellent stories would allow it to be used in classrooms. It also has comparable raw accuracy to average human agreement as shown in Table 6, although it makes more errors than humans in classifying a story off by more than one class off as shown by the statistics  in Table 7. The results most in its favor are shown highlighted in Table 5. It separates with few errors both excellent and good stories from the majority of poor stories.</Paragraph>
    <Paragraph position="1"> While the event calculus captures some of the relevant defining characteristics of stories, it does not capture all of them. The types of stories that give the machine learners the most difficulty are those which are excellent and fair. One reason is that these stories are less frequent in the training data than poor and good stories. Another reason is that there are features particular to these stories that are not accounted for by an event structure or LSA. Both excellent stories and fair stories rely on very subtle features to distinguish them from good and poor stories. Good stories were characterized in the rating criteria as &amp;quot;parroting off of the main events,&amp;quot; and the event calculus naturally is good at identifying this. Poor stories have &amp;quot;definite problems with the recall of events,&amp;quot; and so are also easily identified. However, fair stories show both a lack of &amp;quot;understanding of the point&amp;quot; and &amp;quot;do not really flow&amp;quot; while the excellent story shows an &amp;quot;understanding of the point.&amp;quot; These characteristics involve relations such as the &amp;quot;point&amp;quot; of the story and connections between events. These ideas of &amp;quot;flow&amp;quot; and &amp;quot;point&amp;quot; are much more difficult to analyze automatically.</Paragraph>
  </Section>
class="xml-element"></Paper>