<?xml version="1.0" standalone="yes"?>
<Paper uid="W98-1114">
  <Title>Can Subcategorisation Probabilities Help a Statistical Parser?</Title>
  <Section position="4" start_page="0" end_page="119" type="intro">
    <SectionTitle>
2 Background
</SectionTitle>
    <Paragraph position="0"/>
    <Section position="1" start_page="0" end_page="118" type="sub_section">
      <SectionTitle>
2.1 Subcategorisation Acquisition
</SectionTitle>
      <Paragraph position="0"> Several substantial machine-readable subcategorisation dictionaries exist for English, either built semi-automatically from machine-readable versions of conventional learners' dictionaries, or manually by (computational) linguists (e.g.</Paragraph>
      <Paragraph position="1"> the Alvey NL Tools (ANLT) dictionary, Boguraev et al. (1987); the COMLEX Syntax dictionary, Grishman, Macleod &amp; Meyers (1994)).</Paragraph>
      <Paragraph position="2"> However, since these efforts were not carried out in tandem with rigorous large-scale classification of corpus data, none of the resources produced provide useful information on the relative frequency of different subcategorisation frames.</Paragraph>
      <Paragraph position="3"> Systems which are able to acquire a small number of verbal subcategorisation classes automatically from corpus text have been described by Brent (1991, 1993), and Ushioda et al. (1993). Ushioda et al. also derive relative subcategorisation frequency information for individual predicates. In this work they utilise a part-of-speech (PoS) tagged corpus and finite-state NP parser to recognise and calculate  the relative frequency of six subcategorisation classes. They report that for 32 out of 33 verbs tested their system correctly predicts the most frequent class, and for 30 verbs it correctly predicts the second most frequent class, if there was one.</Paragraph>
      <Paragraph position="4"> Manning (1993) reports a larger experiment, also using a PoS tagged corpus and a finite-state NP parser, attempting to recognise sixteen distinct complementation patterns--although not with relative frequencies. In a comparison between entries for 40 common verbs acquired from 4.1 million words of text and the entries given in the Ozford Advanced Learner's Dictionary off Current English (Hornby, 1989) Manning's system achieves a precision of 90% and a recall of 43%.</Paragraph>
      <Paragraph position="5"> Gahl (1998) presents an extraction tool for use with the BNC that is able to create sub-corpora containing different subcategorisation frames for verbs, nouns and adjectives, given the frames expected for each predicate. The tool is based on a set of regular expressions over PoS tags, lemmas, morphosyntactic tags and sentence boundaries, effectively performing the same function as a chunking parser (c.f. Abney, 1996). The resulting subcorpora can be used to determine the (relative) frequencies of the frames.</Paragraph>
      <Paragraph position="6"> Carroll &amp; Rooth (1998) use an iterative approach to estimate the distribution of subcategorisation frames given head words, starting from a manually-developed context-free grammar (of English). First, a probabilistic version of the grammar is trained from a text corpus using the expectation-maximisation (EM) algorithm, and the grammar is lexicalised on rule heads. The EM algorithm is then run again to calculate the expected frequencies of a head word accompanied by a particular frame.</Paragraph>
      <Paragraph position="7"> These probabilities can then be fed back into the grammar for the next iteration. Carroll &amp; Rooth report encouraging results for three verbs based on applying the technique to text from the BNC.</Paragraph>
      <Paragraph position="8"> Briscoe &amp; Carroll (1997) describe a system capable of distinguishing 160 verbal subcategorisation classes--a superset of those found in the ANLT and COMLEX Syntax dictionaries-returning relative frequencies for each frame found for each verb. The classes also incorporate information about control of predicative arguments and alternations such as particle movement and extraposition. The approach uses a robust statistical parser which yields complete though 'shallow' parses, a comprehensive subcategorisation class classifier, and a priori estimates of the probability of membership of these classes. For a sample of seven verbs with multiple subcategorisation possibilities the system's frequency rankings averaged 81% correct. (We talk about this system further in section 3.2 below, describing how we used it to provide frequency data for our experiment).</Paragraph>
    </Section>
    <Section position="2" start_page="118" end_page="119" type="sub_section">
      <SectionTitle>
2.2 Lexicalised Statistical Parsing
Carroll &amp; Weir (1997)--without actually build-
</SectionTitle>
      <Paragraph position="0"> ing a parsing system--address the issue of how frequency information can be associated with lexicalised grammar formalisms, using Lexicalized Tree Adjoining Grammar (Joshi &amp; Schabes, 1991) as a unifying framework. They consider systematically a number of alternative probao bilistic formulations, including those of Resnik (1992) and Schabes (1992) and implemented systems based on other underlying grammatical frameworks, evaluating their adequacy from both a theoretical and empirical perspective in terms of their ability to model particular distributions of data that occur in existing treebanks. Magerman (1995), Collins (1996), Ratnaparkhi (1997), Charniak (1997) and others describe implemented systems with impressive accuracy on parsing unseen data from the Penn Treebank (Marcus, Santorini &amp; Marcinkiewicz, 1993). These parsers model probabilistically the strengths of association between heads of phrases, and the configurations in which these lexical associations occur. The accuracies reported for these systems are substantially better than their (non-lexicalised) probabilistic context-free grammar analogues, demonstrating clearly the value of lexico-statistical information. However, since the grammatical descriptions are induced from atomic-labeled constituent structures in the training treebank, rather than coming from an explicit generative grammar, these systems do not make contact with traditional notions of argument structure (i.e. subcategorisation, selectional preferences of predicates for complements) in any direct sense.</Paragraph>
      <Paragraph position="1"> So although it is now possible to extract at least subcategorisation data from large corpora 2 with  some degree of reliability, it would be difficult to integrate the data into this type of parsing system.</Paragraph>
      <Paragraph position="2"> Briscoe &amp; Carroll (1997) present a small-scale experiment in which subcategorisation class frequency information for individual verbs w~us integrated into a robust statistical (non-lexicalised) parser. The experiment used a test corpus of 250 sentences, and used the standard GEIG bracket precision, recall and crossing measures (Grishman, Macleod &amp; Sterling, 1992) for evaluation. While bracket precision and recall were virtually unchanged, the crossing bracket score for the lexicalised parser showed a 7% improvement. However, this difference turned out not to be statistically significant at the 95% level: some analyses got better while others got worse.</Paragraph>
      <Paragraph position="3"> We have performed a similar, but much larger scale experiment, which we describe below. We used a larger test corpus, acquired data from an acquisition corpus an order of magnitude larger, and used a different quantitative evaluation measure that we argue is more sensitive to argument/adjunct and attachment distinctions.</Paragraph>
      <Paragraph position="4"> We summarise the main features of the 'baseline' parsing system in section 3.1, describe how we lexicalised it (section 3.2), present the results of the quantitative evaluation (section 3.3), give a qualitative analysis of the analysis errors made (section 3.4), and conclude with directions for future work.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>