<?xml version="1.0" standalone="yes"?>
<Paper uid="P06-1117">
  <Title>Semantic Role Labeling via FrameNet, VerbNet and PropBank</Title>
  <Section position="4" start_page="929" end_page="930" type="metho">
    <SectionTitle>
2 Automatic Semantic Role Labeling
</SectionTitle>
    <Paragraph position="0"> One of the goals of the FN project is to design a linguistic ontology that can be used for the automatic processing of semantic information. The associated hierarchy contains an extensive semantic analysis of verbs, nouns, adjectives and situations in which they are used, called frames. The basic assumption on which the frames are built is that each word evokes a particular situation with specific participants (Fillmore, 1968). The word that evokes a particular frame is called target word or predicate and can be an adjective, noun or verb.</Paragraph>
    <Paragraph position="1"> The participant entities are defined using semantic roles and they are called frame elements.</Paragraph>
    <Paragraph position="2"> Several models have been developed for the automatic detection of the frame elements based on the FN corpus (Gildea and Jurafsky, 2002; Thompson et al., 2003; Litkowski, 2004). While the algorithms used vary, almost all the previous studies divide the task into: 1) the identification of the verb arguments to be labeled and 2) the tagging of each argument with a role. Also, most of the models agree on the core features as being: Predicate, Headword, Phrase Type, Governing Category, Position, Voice and Path. These are the initial features adopted by Gildea and Jurafsky (2002) (henceforth G&amp;J) for both frame element identification and role classification.</Paragraph>
    <Paragraph position="3"> One difference among previous machine-learning models is whether they used the frame information or not. The impact of the frame feature over unseen predicates and words is particularly interesting for us. The results obtained by G&amp;J provide some interesting insights in this direction.</Paragraph>
    <Paragraph position="4"> In one of their experiments, they used the frame to generalize from predicates seen in the training data to unseen predicates, which belonged to the same frame. The overall performance increased showing that when no training data is available for a target word we can use data from the same frame.</Paragraph>
    <Paragraph position="5"> Other studies suggest that the frame is crucial when trying to eliminate the major sources of errors. In their error analysis, (Thompson et al., 2003) pinpoints that the verb arguments with headwords that are rare in a particular frame but not rare over the whole corpus are especially hard to classify. For these cases the frame is very important because it provides the context information needed to distinguish between different word senses.</Paragraph>
    <Paragraph position="6"> Overall, the experiments presented in G&amp;J's study correlated with the results obtained in the Senseval-3 competition show that the frame feature increases the performance and decreases the amount of annotated examples needed in training (i.e. frame usage improves the generalization ability of the learning algorithm). On the other hand, the results obtained without the frame information are very poor.</Paragraph>
    <Paragraph position="7"> These results show that having broader frame coverage is very important for robust semantic parsing. Unfortunately, the 321 frames that contain at least one verb predicate cover only a small fraction of the English verb lexicon and of the possible domains. Also from these 321 frames only 100 were considered to have enough training data and were used in Senseval-3 (see (Litkowski, 2004) for more details).</Paragraph>
    <Paragraph position="8"> Our approach for solving such problems involves the usage of a frame-like feature, namely the Intersective Levin class (ILC). We show that the ILC can replace the frame with almost no loss in performance. At the same time, ILC provides better coverage as it can be learned also from other  corpora (e.g. PB).</Paragraph>
    <Paragraph position="9"> The next section provides the theoretical support for the unified usage of FN, VN and PB, explaining why and how it is possible to link them.</Paragraph>
  </Section>
  <Section position="5" start_page="930" end_page="931" type="metho">
    <SectionTitle>
3 Linking FrameNet to VerbNet and
PropBank
</SectionTitle>
    <Paragraph position="0"> In general, predicates belonging to the same FN frame have a coherent syntactic behavior that is also different from predicates pertaining to other frames (G&amp;J). This finding is consistent with theories of linking that claim that the syntactic behavior of a verb can be predicted from its semantics (Levin, 1993). This insight justifies the attempt to use ILCs instead of the frame feature when classifying FN semantic roles (Giuglea and Moschitti, 2004).</Paragraph>
    <Paragraph position="1"> The main advantage of using Levin classes comes from the fact that other resources like PB and the VN lexicon contain this kind of information. Thus, we can train an ILC classifier also on the PB corpus, considerably increasing the verb knowledge base at our disposal. Another advantage derives from the syntactic criteria that were applied in defining the Levin's clusters. As shown later in this article, the syntactic nature of these classes makes them easier to classify than frames when using only syntactic and lexical features.</Paragraph>
    <Paragraph position="2"> More precisely, Levin's clusters are formed according to diathesis alternation criteria which are variations in the way verbal arguments are grammatically expressed when a specific semantic phenomenon arises. For example, two different types of diathesis alternations are the following:  (a) Middle Alternation [Subject, Agent The butcher] cuts [Direct Object, Patient the meat].</Paragraph>
    <Paragraph position="3"> [Subject, Patient The meat] cuts easily.</Paragraph>
    <Paragraph position="4"> (b) Causative/inchoative Alternation  [Subject, Agent Janet] broke [Direct Object, Patient the cup].</Paragraph>
    <Paragraph position="5"> [Subject, Patient The cup] broke.</Paragraph>
    <Paragraph position="6"> In both cases, what is alternating is the grammatical function that the Patient role takes when changing from the transitive use of the verb to the intransitive one. The semantic phenomenon accompanying these types of alternations is the change of focus from the entity performing the action to the theme of the event.</Paragraph>
    <Paragraph position="7"> Levin documented 79 alternations which constitute the building blocks for the verb classes. Although alternations are chosen as the primary means for identifying the classes, additional properties related to subcategorization, morphology and extended meanings of verbs are taken into account as well. Thus, from a syntactic point of view, the verbs in one Levin class have a regular behavior, different from the verbs pertaining to other classes. Also, the classes are semantically coherent and all verbs belonging to one class share the same participant roles.</Paragraph>
    <Paragraph position="8"> This constraint of having the same semantic roles is further ensured inside the VN lexicon which is constructed based on a more refined version of the Levin's classification, called Intersective Levin classes (ILCs) (Dang et al., 1998). The lexicon provides a regular association between the syntactic and semantic properties of each of the described classes. It also provides information about the syntactic frames (alternations) in which the verbs participate and the set of possible semantic roles.</Paragraph>
    <Paragraph position="9"> One corpus associated with the VN lexicon is PB. The annotation scheme of PB ensures that the verbs belonging to the same Levin class share similarly labeled arguments. Inside one ILC, to one argument corresponds one semantic role numbered sequentially from ARG0 to ARG5. The adjunct roles are labeled ARGM.</Paragraph>
    <Paragraph position="10"> Levin classes were constructed based on regularities exhibited at grammatical level and the resulting clusters were shown to be semantically coherent. As opposed, the FN frames were built on semantic bases, by putting together verbs, nouns and adjectives that evoke the same situations. Although different in conception, the FN verb clusters and VN verb clusters have common properties1: null  1. Different syntactic properties between distinct verb clusters (as proven by the experiments in G&amp;J) 2. A shared set of possible semantic roles for all  verbs pertaining to the same cluster.</Paragraph>
    <Paragraph position="11"> Having these insights, we have assigned a correspondent VN class not to each verb predicate but rather to each frame. In doing this we have applied the simplifying assumption that a frame has a  unique corresponding Levin class. Thus, we have created a one-to-many mapping between the ILCs and the frames. In order to create a pair&lt;FN frame, VN class&gt; , our mapping algorithm checks both the syntactic and semantic consistency by comparing the role frequency distributions on different syntactic positions for the two candidates. The algorithm is described in detail in the next section.</Paragraph>
  </Section>
  <Section position="6" start_page="931" end_page="932" type="metho">
    <SectionTitle>
4 Mapping FrameNet frames to VerbNet
</SectionTitle>
    <Paragraph position="0"> classes The mapping algorithm consists of three steps: (a) we link the frames and ILCs that have the largest number of verbs in common and we create a set of pairs &lt;FN frame, VN class&gt; (see Table 1); (b) we refine the pairs obtained in the previous step based on diathesis alternation criteria, i.e. the verbs pertaining to the FN frame have to undergo the same diathesis alternation that characterize the corresponding VN class (see Table 2) and (c) we manually check the resulting mapping.</Paragraph>
    <Section position="1" start_page="931" end_page="932" type="sub_section">
      <SectionTitle>
4.1 The mapping algorithm
</SectionTitle>
      <Paragraph position="0"> Given a frame, F, we choose as candidate for the mapping the ILC, C, that has the largest number of verbs in common with it (see Table 1, line (I)). If the number is greater or equal than three we form a pair &lt;F, C&gt; that will be tested in the second step of the algorithm. Only the frames that have more than 3 verb lexical units are candidates for this step (frames with less than 3 members cannot pass condition (II)). This excludes a number of 60 frames that will be subsequently manually mapped.</Paragraph>
      <Paragraph position="1"> In order to assign a VN class to a frame, we have to verify that the verbs belonging to the FN frame participate in the same diathesis alternation criteria used to define the VN class. Thus, the pairs &lt;F,C&gt; formed in step 1 of the mapping algorithm have to undergo a validation step that verifies the similarity between the enclosed FN frame and VN class. This validation process has several sub-steps: First, we make use of the property (2) of the Levin classes and FN frames presented in the previous section. According to this property, all verbs pertaining to one frame or ILC have the same participant roles. Thus, a first test of compatibility between a frame and a Levin class is that they share the same participant roles. As FN is annotated with frame-specific semantic roles, we manually mapped these roles into the VN set of the-</Paragraph>
      <Paragraph position="3"/>
      <Paragraph position="5"> matic roles. Given a frame, we assigned thematic roles to all frame elements that are associated with verbal predicates. For example the Speaker, Addressee, Message and Topic roles from the Telling frame were respectively mapped into the Agent, Recipient, Theme and Topic theta roles.</Paragraph>
      <Paragraph position="6"> Second, we build a frequency distribution of VN thematic roles on different syntactic positions.</Paragraph>
      <Paragraph position="7"> Based on our observation and previous studies (Merlo and Stevenson, 2001), we assume that each ILC has a distinct frequency distribution of roles on different grammatical slots. As we do not have matching grammatical functions in FN and VN, we approximate that subjects and direct objects are more likely to appear on positions adjacent to the predicate, while indirect objects appear on more distant positions. The same intuition is successfully used by G&amp;J to design the Position feature. null For each thematic role thi we acquired from VN and FN data the frequencies with which thi appears on an adjacent A or distant D positions in a given frame or VN class (i.e. #&lt;thi, class, position&gt; ).</Paragraph>
      <Paragraph position="8"> Therefore, for each frame and class, we obtain two vectors with thematic role frequencies corresponding respectively to the adjacent and distant positions (see Table 2). We compute a score for each  pair &lt;F,C&gt; using the normalized scalar product.</Paragraph>
      <Paragraph position="9"> The core arguments, which tend to occupy adjacent positions, show a minor syntactic variability and are more reliable than adjunct roles. To account for this in the overall score, we multiply the adjacent and the distant scores by 2/3 and 1/3, respectively. This limits the impact of adjunct roles like Temporal and Location.</Paragraph>
      <Paragraph position="10"> The above frequency vectors are computed for FN directly from the corpus of predicate-argument structure examples associated with each frame.</Paragraph>
      <Paragraph position="11"> The examples associated with the VN lexicon are extracted from the PB corpus. In order to do this we apply a preprocessing step in which each label Arg0..5 is replaced with its corresponding thematic role given the ILC of the predicate. We assign the same roles to the adjuncts all over PB as they are general for all verb classes. The only exception is ARGM-DIR that can correspond to Source, Goal or Path. We assign different roles to this adjunct based on the prepositions. We ignore some adjuncts like ARGM-ADV or ARGM-DIS because they cannot bear a thematic role.</Paragraph>
    </Section>
    <Section position="2" start_page="932" end_page="932" type="sub_section">
      <SectionTitle>
4.2 Mapping Results
</SectionTitle>
      <Paragraph position="0"> We found that only 133 VN classes have correspondents among FN frames. Moreover, from the frames mapped with an automatic score smaller than 0.5 almost a half did not match any of the existing VN classes2. A summary of the results is depicted in Table 3. The first column contains the automatic score provided by the mapping algorithm when comparing frames with ILCs. The second column contains the number of frames for each score interval. The third column contains the percentage of frames that did not have a corresponding VN class and finally the fourth and fifth columns contain the accuracy of the mapping algorithm for each interval score and for the whole task, respectively.</Paragraph>
      <Paragraph position="1"> We mention that there are 3,672 distinct verb senses in PB and 2,351 distinct verb senses in 2The automatic mapping is improved by manually assigning the FN frames of the pairs that receive a score lower than 0.5.</Paragraph>
      <Paragraph position="2"> FN. Only 501 verb senses are in common between the two corpora which means 13.64% of PB and 21.31% of FN. Thus, by training an ILC classifier on both PB and FN we extend the number of available verb senses to 5,522.</Paragraph>
    </Section>
    <Section position="3" start_page="932" end_page="932" type="sub_section">
      <SectionTitle>
4.3 Discussion
</SectionTitle>
      <Paragraph position="0"> In the literature, other studies compared the Levin classes with the FN frames, e.g. (Baker and Ruppenhofer, 2002; Giuglea and Moschitti, 2004; Shi and Mihalcea, 2005). Their findings suggest that although the two set of clusters are roughly equivalent there are also several types of mismatches:  1. Levin classes that are narrower than the corresponding frames, 2. Levin classes that are broader that the corresponding frames and 3. Overlapping groups.</Paragraph>
      <Paragraph position="1">  For our task, point 2 does not pose a problem. Points 1 and 3 however suggest that there are cases in which to one FN frame corresponds more than one Levin class. By investigating such cases, we noted that the mapping algorithm consistently assigns scores below 75% to cases that match problem 1 (two Levin classes inside one frame) and below 50% to cases that match problem 3 (more than two Levin classes inside one frame). Thus, to increase the accuracy of our results, a first step should be to assign independently an ILC to each of the verbs pertaining to frames with score lower than 0.75%.</Paragraph>
      <Paragraph position="2"> Nevertheless the current results are encouraging as they show that the algorithm is achieving its purpose by successfully detecting syntactic incoherences that can be subsequently corrected manually. Also, in the next section we will show that our current mapping achieves very good results, giving evidence for the effectiveness of the Levin class feature.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>