<?xml version="1.0" standalone="yes"?>
<Paper uid="W98-1113">
  <Title>Towards Unsupervised Extraction of Verb Paradigms from Large Corpora</Title>
  <Section position="7" start_page="114" end_page="116" type="concl">
    <SectionTitle>
6 Results
</SectionTitle>
    <Paragraph position="0"> While our goal was to use a supervised method to evaluate the performance of the unsupervised method, the supervised functions we tested differed widely in their ability to predict the optimum cut point for the left and right context trees. The performance of the gain ratio, revised ratio, and percent correct are compared to the unsupervised method on left and right context cluster trees in Figures 4 and 5. The x axis gives the height at which the cut is evaluated by the function, and the y axis is the scaled value of the function for that height. The optimum cut point indicated by each function is at the height for which the function has the maximum value. These heights are given in Table 3. For the right context tree, for which the optimum cut produces many small clusters, there is general agreement between the unsupervised and supervised methods. For the left context tree, for which the optimum cut produces a few large clusters, there is a lack of agreement among the supervised methods with the gain ratio failing to indicate a meaningful cut. The maximum for the unsupervised method falls between the maxima for the revised ratio and percent correct. Based on these results, we used the unsupervised maximum to select the cut point for the left and right context cluster trees.</Paragraph>
    <Paragraph position="1"> There are four steps in the analysis of the data. First, the cutpoint for the left context  tree is determined. Second, the right similarity matrix is enhanced with data from left context clusters. Third, the cut point for the enhanced right context tree is determined. Finally, the verbs are cross-classified by left and right context cluster membership.</Paragraph>
    <Paragraph position="2"> Step 1: We select the cut point for the left context tree at height T = 18, the unsupervised maximum. This cut creates clusters that axe 90.7% correct as compared to 91.5% at height T = 15. At T = 18 there are 20 clusters for 6 inflectional categories, of which 6 are in the size range 20-180 specified by the unsupervised method.</Paragraph>
    <Paragraph position="3"> Step 2: Reasoning that two verbs in the same cluster for inflectional category should be in different clusters for lemmas, 7 we created a new similarity matrix for the right context by increasing the distance between each pair of verbs that occurred in the same left context cluster.</Paragraph>
    <Paragraph position="4"> The distance was increased by substituting a constant equal to the value of the maximum distance between verbs. The number of verbs correctly classified increased from 63.5% for the original right context tree to 74.7% for the enhanced right context tree.</Paragraph>
    <Paragraph position="5"> Step3: We select the cut point for the enhanced right context tree at height T = 12, the unsupervised maximum. This cut creates clusters that are 72.2% correct as compared to 74.7% at height T = 10. There are 155 clusters at height T = 12, which is 21 less than the total number of lemmas in the data set. 29% of the clusters are singletons which is lower than the proportion of singleton lemmas (45%). 60% of the singleton clusters contain singleton lemmas and are counted as correct.</Paragraph>
    <Paragraph position="6"> Step ~: Each left context cluster was given a unique left ID and labeled for the dominant inflectional category. Each right context cluster was given a right ID and labeled for the dominant lemma. By identifying the left and  right cluster membership for each verb, we were able to predict the correct inflectional category and lemma for 67.5% of the 400 verbs. Table 4 shows a set of consecutive right clusters in which the lemma and inflectional category are co.rrectly predicted for all but one verb.</Paragraph>
    <Section position="1" start_page="115" end_page="116" type="sub_section">
      <SectionTitle>
6.1 Conclusion
</SectionTitle>
      <Paragraph position="0"> We have clearly demonstrated that a surprising amount of information about the verb paradigm is strictly local to the verb. We believe that distributional information combined with morphological information will produce extremely accurate classifications for both regular and irregular verbs. The fact that information consisting of nothing more than bigrams can capture syntactic information about English has already been noted by (Brown et al. 1992).</Paragraph>
      <Paragraph position="1"> Our contribution has been to develop a largely unsupervised method for cutting a cluster tree that produces reliable classifications. We also developed an unsupervised method to enhance the classification of one cluster tree by using the classification of another cluster tree. The verb paradigm is extracted by cross classifying a verb by lemma and inflectional category. This method depends on the successful classification of verbs by lemma and inflectional category sep- null arately.</Paragraph>
      <Paragraph position="2"> We are encouraged by these results to continue working towards fully unsupervised methods for extracting verb paradigms using distributional information. We hope to extend this exploration to other languages. We would also like to explore how the dual mechanisms of encoding verb lemmas and inflectional categories both by distributional criteria and by morphology can be exploited both by the child language learner and by automatic grammar extraction processes.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>