<?xml version="1.0" standalone="yes"?>
<Paper uid="J00-4004">
  <Title>Learning Methods to Combine Linguistic Indicators: Improving Aspectual Classification and Revealing Linguistic Insights</Title>
  <Section position="10" start_page="623" end_page="624" type="concl">
    <SectionTitle>
9. Conclusions
</SectionTitle>
    <Paragraph position="0"> While individual linguistic indicators have predictive value, they are predictively incomplete. Such incompleteness is due to sparsity and noise when computing indicator values over a corpus of limited size, and is also a consequence of the linguistic behavior of certain indicators. However, incomplete indicators can complement one another when placed in combination.</Paragraph>
    <Paragraph position="1"> Machine learning has served to illustrate the potential of 14 linguistic indicators by showing that they perform well in combination for two aspectual classification problems. This potential was not clear when evaluating indicators individually. For stativity, decision tree induction achieved an accuracy of 93.9%, as compared to the uninformed baseline's accuracy of 83.8%. Furthermore, GP and logistic regression also achieved improvements over the baseline. For completedness, CART and logistic regression achieved accuracies of 74.0% and 70.5%, as compared to the uninformed baseline's accuracy of 63.3%. These improvements in classification performance are more dramatically illustrated by favorable trade-offs between recall scores achieved for both classification problems. Such results are profitable for tasks that weigh the identification of the less frequent class more heavily.</Paragraph>
    <Paragraph position="2"> This evaluation was performed over unrestricted sets of verbs occurring across two corpora. The system can automatically classify all verbs appearing in a corpus, including those that have not been manually classified for supervised training data.</Paragraph>
    <Paragraph position="3"> Therefore, we have demonstrated a much-needed full-scale method for aspectual classification that is readily expandable. Since minimal overfitting was demonstrated with only a small quantity of manually supervised data required, this approach is easily portable to other domains, languages, and semantic distinctions.</Paragraph>
    <Paragraph position="4"> The results of learning are linguistically viable in two respects. First, learning automatically produces models that are specialized for different aspectual distinctions; the same set of 14 indicators are combined in different ways according to which classification problem is targeted. Second, automatic learning often derives linguistically informative insights. We have shown several such insights revealed by inspecting the models produced by learning, which are summarized here: * Examining the logistic regression model for classification according to stativity revealed a decision tree type of rule incorporated with the normal weighting scheme.</Paragraph>
    <Paragraph position="5"> * Verb frequency distinguishes stative verbs within multiple subsets of verbs. When applied to all verbs in a medical corpus, it identifies occurrences of show. Furthermore, examining an example node of the decision tree that distinguishes according to stativity revealed that verb frequency discriminates 19 stative clauses with 100.0% precision from the node's partition of 60 training cases.</Paragraph>
    <Paragraph position="6"> * Several proper subsets of the linguistic indicators prove independently useful for aspectual classification when combined with an appropriate model. This is illustrated by the fact that certain models reveal combinations of small sets of indicators that improved classification performance. For example, GP results for both classification tasks  Siegel and McKeown Improving Aspectual Classification incorporated a subset of only five indicators each. In particular, manner adverb, which ranked highest by logistic regression, is not incorporated in the example function tree induced by GP. This may be because this indicator only applies to a small number of verbs, as shown in Table 13, and because an/f-rule such as that captured by logistic regression is difficult to encode with a function tree with no conditional primitives.</Paragraph>
    <Paragraph position="7"> Learning methods discovered that some indicators are particularly useful for both classification tasks. For example, the same two indicators were weighted most heavily by logistic regression for both tasks: duration in-PP and manner adverb.</Paragraph>
    <Paragraph position="8"> However, in general, learning methods emphasized different linguistic indicators for different classification tasks. For example, decision tree induction used frequency as the main discriminator to classify clauses according to stativity, while the perfect indicator was the main discriminator for classification according to completedness.</Paragraph>
    <Paragraph position="9"> Comparing the ability of learning methods to combine linguistic indicators is difficult, since they rank differently depending on the classification task and evaluation criteria. For example, the relative accuracies of the three supervised learning procedures rank in opposite orders when comparing the results for stativity to the results for completedness.</Paragraph>
    <Paragraph position="10"> The unsupervised grouping of verbs provides an additional method for aspectual classification according to stativity. Co-occurrence distributions between the verb and direct object inform the aspectual classification of verbs. This provides information beyond the 14 linguistic indicators that can also be derived automatically. However, due to the sparsity intrinsic to pairs of open-class categories such as verb-object pairs, this approach was only evaluated over a small set of frequent verbs.</Paragraph>
  </Section>
class="xml-element"></Paper>