<?xml version="1.0" standalone="yes"?>
<Paper uid="C02-1132">
  <Title>Probabilistic Models of Verb-Argument Structure</Title>
  <Section position="3" start_page="0" end_page="0" type="metho">
    <SectionTitle>
2 Capturing Alternation Behavior
</SectionTitle>
    <Paragraph position="0"> Automatic clustering of co-occurrences of verbs and their direct objects was first used to induce semantically related classes of both verbs and nouns (Pereira et al., 1993). Rooth et al. (1999) used the Expectation Maximization algorithm to perform soft clustering by optimizing the parameters of a fairly simple probability model, which considers the verb and noun to be independent given the unobserved cluster variable CR:</Paragraph>
    <Paragraph position="2"> In Rooth et al. (1999), the variable DA represented not only the lexical verb but also its syntactic relation to the noun: either direct object, subject of an intransitive, or subject of a transitive verb.</Paragraph>
    <Paragraph position="3"> However, the relationship between the underlying, semantic arguments of a verb and the syntactic roles in a sentence is not always straightforward. Many verbs exhibit alternations in their syntactic behavior, as shown by the following examples:  (1) The Federal Reserve increased rates by 1/4%.</Paragraph>
    <Paragraph position="4"> (2) Interest rates have increased sharply over the  past year.</Paragraph>
    <Paragraph position="5"> The noun rates appears as the syntactic object of the verb increase in the first sentence, but as its subject in the second sentence, where the verb is used intransitively, that is, without an object. One of the clusters found by the model of Rooth et al. (1999) corresponded to &amp;quot;verb of scalar change&amp;quot; such as increase, rise,anddecrease.The model places both subject-of-intransitive-increase and direct-object-of-increase in this class, but does not explicitly capture the fact that these to values represent different uses of the same verb.</Paragraph>
    <Paragraph position="6"> The phenomenon of verb argument alternations has been most comprehensively studied by Levin (1993), who catalogs over 3,000 verbs into classes according to which alternations they participate in. A central thesis of Levin's work is that a verb's syntactic alternations are related to its semantics, and that semantically related verb will share the same alternations. For example, the alternation of examples 1 and 2 is shared by verbs such as decrease and diminish.</Paragraph>
    <Paragraph position="7"> Table 1 gives the most common nouns occurring as arguments of selected verbs in our corpus, showing how alternation behavior shows up in corpus statistics. The verbs open and increase, classified by Levin and others as exhibiting a causative alternation between transitive and intransitive usages, share many of the same nouns in direct object and subject-of-intransitive positions, as we would expect. For example, number, cost,andrate occur among the ten most common nouns in both positions for increase, and themselves seem semantically related. For open, the first three words in either position are the same. For the verb play, on the other hand, classified as an &amp;quot;object-drop&amp;quot; verb by Merlo and Stevenson (2001), we would expect overlap between the subject of transitive and intransitive uses. This is in fact the case, with child, band,andteam appearing among the top ten nouns for both positions. However, play also exhibits an alternation between the direct object and subject of intransitive positions for music, role,andgame. These two sets of nouns seem to fill different semantic roles of the verb, the first set being agents and the second being themes. This example illustrate the complex interaction between verb sense and alternation behavior: &amp;quot;The band played&amp;quot; and the &amp;quot;The music played&amp;quot; are considered to belong to different senses of play by WordNet (Fellbaum, 1998) and other word sense inventories. However, it is interesting to note that nouns from both the broad senses of play,&amp;quot;playa game&amp;quot; and &amp;quot;play music&amp;quot;, participate in both alternations. An advantage of our EM-based soft clustering algorithm is that it can assign a verb to multiple clusters; ideally, we would hope that a verb's clusters would correspond to its senses.</Paragraph>
    <Paragraph position="8"> We expect verbs which take similar sets of argument fillers to be semantically related, and to participate in the same alternations. This idea has been used by McCarthy (2000) to identify verbs participating in specific alternations by looking for overlap between nouns used in different positions, and by using WordNet to classify role fillers into semantic categories. Schulte im Walde (2000) uses an EM-based automatic clustering of verbs to attempt to derive Levin classes from unlabeled data.</Paragraph>
    <Paragraph position="9"> As in McCarthy (2000), the nouns are classified using WordNet. However, the appearance of the same noun in different syntactic positions is not explicitly captured by the probability model used for clustering. null This observation motivated a new probabilistic model of verb argument structure designed to explicitly capture alternation behavior. In addition to an unobserved cluster variable CR, we introduce a second unobserved variable D6 for the semantic role of an argument. The role D6 is dependent on both the cluster CR to which our verb-noun pair belongs, and the syntactic slot D7 in which the noun is found, and the probability of an observed triple C8B4DABND7BND2B5 is estimated as:</Paragraph>
    <Paragraph position="11"> The noun is independent of the verb given the cluster variable, as before, and the noun is independent of the syntactic slot D7 given the cluster CR and the semantic role D6. The semantic role variable D6 can take two values, with C8B4D6CYCRBND7B5 representing the mapping from syntax to semantic role for a cluster of verbs.</Paragraph>
    <Paragraph position="12"> We expect the clusters to consist of verbs that not only appear with the same set of nouns, but share the same mapping from syntactic position to semantic role. For example increase and decrease might belong to same cluster as they both appear frequently Verb Object Subj of Intransitive Subj of Transitive close door door troop eyes eyes door mouth mouth police firebreak exhibition gunman way shop woman possibility show man gate trial guard account conference soldier window window one shop gate company increase risk number government number proportion increase share population use profit rate effect lead pressure sale pressure amount level rate cost presence likelihood sale Party chance rates Labour cost profit bank play part child band role band factor  with rate, number,andprice in both the direct object and subject of intransitive slots, and would assign the same value of D6 to both positions. The verb lower might belong to a different cluster because, although it appears with the same nouns, they appear as the direct object but not as the subject. The Expectation Maximization algorithm is used to train the model from the corpus, iterating over an Expectation step in which expected values for the two unobserved variables CR and D6 are calculated for each observation in the training data, and a Maximization step in which the parameter of each of the five distributions C8B4CRB5, C8B4DACYCRB5, C8B4D7CYCRB5, C8B4D6CYCRBND7B5, and C8B4D2CYD2BNCRB5 are set to maximize the likelihood of the data given the expectations for CR and D6.</Paragraph>
    <Section position="1" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
3TheData
</SectionTitle>
      <Paragraph position="0"> For our experiments we used a version of the British National Corpus parsed with the statistical parser of Collins (1997). Subject and direct object relations were extracted by searching for NP nodes dominated by S and VP nodes respectively. The head words of the resulting subject and object nodes were found using the deterministic headword rules employed by the parsing model. The individual observations of our dataset are noun-verb pairs of three types: direct object, subject of a verb with an object, and subject of a verb without an object. As a result, the subject and object relations of the same original sentence are considered independently by all of the models we examine.</Paragraph>
      <Paragraph position="1"> Direct object noun phrases were assigned the function tags of the Treebank-2 annotation style (Marcus et al., 1994) in order to distinguish noun phrases such as temporal adjuncts from true direct objects. For example, in the sentence &amp;quot;He ate yesterday&amp;quot;, yesterday would be assigned the Temporal tag, and therefore not considered a direct object for our purposes. Similarly, in the sentence &amp;quot;Interest rates rose 2%&amp;quot;, 2% would be assigned the Extent tag, and this instance of rise would be considered intransitive.</Paragraph>
      <Paragraph position="2"> Function tags were assigned using a simple probability model trained on the Wall Street Journal data from the Penn Treebank, in a technique similar to that of Blaheta and Charniak (2000). The model predicts the function tag conditioned on the verb and head noun of the noun phrase:  where CU ranges over the function tags defined (Marcus et al., 1994), or the null tag. Only cases assigned the null tag by this model were considered true direct objects. Evaluated on the binary task of whether to assign a function tag to noun phrases in object position, this classifier was correct 95% of the time on held-out data from the Wall Street Journal. By never assigning a function tag, one would achieve 85% accuracy. While we have no way to evaluate its accuracy on the British National Corpus, certain systematic errors are apparent. For example, while it classifies 2% as an Extent in &amp;quot;Interest rates increased 2%&amp;quot;, it assigns no tag to crack in &amp;quot;The door opened a crack&amp;quot;. This type of error leads to the appearance of door as a subject on transitive uses of open in Table 1.</Paragraph>
      <Paragraph position="3"> Both verbs and nouns were lemmatized using the XTAG morphological dictionary (XTAG Research Group, 2001). As we wished to focus on alternation behavior, verbs that were used intransitively than 90% of the time were excluded from the data; we envision that they would be handled by a separate probability model. Pronouns were excluded from the dataset, as were verbs and nouns that occurred fewer than 10 times, resulting in a vocabulary of 4,456 verbs and 17,345 nouns. The resulting dataset consisted of 1,372,111 triples of verb, noun, and syntactic relation. Of these, 90% were used as training material, 5% were used as a cross-validation set for setting linear interpolation and deterministic annealing parameters, and 5% were used as test data for the results reported below.</Paragraph>
    </Section>
  </Section>
  <Section position="4" start_page="0" end_page="0" type="metho">
    <SectionTitle>
4 The Models
</SectionTitle>
    <Paragraph position="0"> We compare performance of a number of probability models for our verb argument data in order to explore the dependencies of the data and the impact of clustering. Graphical representations of the clustering models are shown in Figure 1.</Paragraph>
    <Paragraph position="1"> Unigram Baseline: This model assumes complete independence of the verb, syntactic slot, and noun, and serves to provide a baseline for the complexity of the task:</Paragraph>
    <Section position="1" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
Three-way Aspect: Following Hofmann and
</SectionTitle>
      <Paragraph position="0"> Puzicha (1998), we refer to EM-based clustering as the aspect model, where different values of the cluster variable are intended to represent abstract &amp;quot;aspects&amp;quot; of the data. The simplest version of the clustering model predicts verb, slot, and noun independently given the cluster  Alternation: This model, described in more detail above, introduces a new unobserved variable D6 for the semantic role of the noun, which can take two values:  Fixed Alternation: This model is designed to incorporate the assumption that the semantic roles of the subject and object of the same verb must be different. The independence assumptions are identical to those of the simple alternation model:</Paragraph>
      <Paragraph position="2"> but the probability C8B4D6CYD7BNCRB5 is only trained for D7 BP subj-intrans. The model is constrained to assign one value of the role variable to direct objects, C8B4D6 BP BCCYD7 BP objB5 BP BD and the other role to subjects of transitives: C8B4D6 BP BDCYD7 BP subj-transB5BPBD.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>