<?xml version="1.0" standalone="yes"?>
<Paper uid="C02-1040">
  <Title>Learning Verb Argument Structure from Minimally Annotated Corpora</Title>
  <Section position="5" start_page="3" end_page="3" type="evalu">
    <SectionTitle>
5 Results
</SectionTitle>
    <Paragraph position="0"> We tried all possible feature combinations (individual features and all possible conjunctions of those features) to explore the contributions of each feature to the reduction of the error rate. The following are the results of the best performing feature combinations. null With our base features, ACT, PASS, VBD, VBN, TRAN, and INTRAN we get the average error rate of 49.4% for 10 fold cross validation. We can see that when we add the CAUS feature, the average error decreases to 41.1%. The CAUS feature helps in decreasing the error rate. Also, when we add the ANIM feature, we get a much better performance. Our average error rate decreases to 37.5%.  the classifier.</Paragraph>
    <Paragraph position="1"> This is the lowest error rate we can achieve by adding one extra feature in addition to the base features. The ANIM feature is an important feature that we can use to construct the classifier. When we add the PART OF SPEECH feature, the error rate also decreases to 39.2%. Therefore, the PART OF SPEECH also helps reduce the error rate as well.</Paragraph>
    <Paragraph position="2"> When we put together the CAUS feature and ANIM feature, we achieve the lowest error rate, which is 33.4%. When we put the PART OF SPEECH and CAUS features together, the error rate does not really decrease (39.0%), comparing to the result with only the PART OF SPEECH feature. The reason of this result should be that there are some parts of the PART OF SPEECH feature and CAUS feature that overlap. When we add the ANIM and PART OF SPEECH features together, the error rate does decrease to 35.8%. Although the result is not as good as result of using ANIM and CAUS features, the combination of the ANIM and PART OF SPEECH features could be considered e ective features that we can use to construct the classifier. We then combine all the features together. The result as expected is not very good. The error rate is 39.5%. The reason should be the same reason as the lower performance when combining the CAUS and PART OF SPEECH features.</Paragraph>
    <Paragraph position="3"> Note that the features TRAN/INTRAN are needed for computing a large subset of the features used. Hence we did not conduct any experiments without these features. These experiments show that the use of SF learning can be useful to the performance of the verb alternation classifier. The error rate of the baseline classifier (picking the right argument structure at chance) was 65.5%. (Merlo and Stevenson, 2001) calculate the expert-based upper bound at this task to be an error rate of 13.5%.</Paragraph>
    <Paragraph position="4"> Our best performing classifier achieves a 33.4% error rate. In comparison, (Merlo and Stevenson, 2001) obtain an error rate of 30.2% using a tagged and automatically parsed data set of 65M words of WSJ text. Thus, while we obtain a slightly worse error rate, this is obtained using a much smaller set of training data.</Paragraph>
  </Section>
class="xml-element"></Paper>