<?xml version="1.0" standalone="yes"?>
<Paper uid="W04-3214">
  <Title>The Influence of Argument Structure on Semantic Role Assignment</Title>
  <Section position="7" start_page="0" end_page="0" type="concl">
    <SectionTitle>
6 Conclusion and Outlook
</SectionTitle>
    <Paragraph position="0"> In this paper, we have performed an error analysis for semantic role assignment, concentrating on the relationship between argument structure and semantic role assignment. To obtain general results, we kept our models as general as possible and verified our results in two different statistical frameworks.</Paragraph>
    <Paragraph position="1"> In our first experiment, we showed that there is considerable variance across frames in the performance of semantic role assignment, and hypothesised that the effect was due to the varying &amp;quot;difficulty&amp;quot; of the underlying argument structure. To test the hypothesis, we defined a measure of frame uniformity which modelled the variability of argument structure. In a second experiment, in which we controlled for other plausible sources of variance, we showed a reliable correlation between performance and uniformity figures.</Paragraph>
    <Paragraph position="2"> The underlying reason for the difficulty of semantic role assignment is that FrameNet is essentially an ontological classification. While the predicates of one frame share the same semantic arguments, they can vary widely in their linking patterns. Without unlimited training data, automatic role assignment has to find and exploit regularities in linking to achieve good results. A priori, this can only be done within frames, since roles are frame-specific, and there is no unique right mapping between roles.</Paragraph>
    <Paragraph position="3"> Consequently, as observed by Fleischman et al. (2003), relatively rare constructions, such as passives, are frequent error sources. Because such constructions have to be learnt individually for each frame, data sparseness is a serious issue. A similar problem arises for lexical differences in the linking properties of predicates in a frame, as with the collide vs. strike case discussed above. Here, the learning has to take into account that the relevant linking properties differ between individual predicates.</Paragraph>
    <Paragraph position="4"> Our results suggest that the variance caused by argument structure will not disappear with better classifiers, but that the problem of inadequate generalisations should be addressed in a principled way.</Paragraph>
    <Paragraph position="5"> There are several possible approaches to do so.</Paragraph>
    <Paragraph position="6"> First, the classic statistical approach: Combining evidence from different frame-specific roles to alleviate data sparseness. To this end, Gildea and Jurafsky (2002) developed a mapping from frame-specific to syntactic roles, but results did not improve much. Baldewein et al. (2004) experiment with EM-driven generalisation, and obtain also only modest improvements.</Paragraph>
    <Paragraph position="7"> A second approach is to identify other levels, different from frames, at which regularities can be learnt better. One possibility is to identify smaller units within frames which have a more uniform structure and which can be learnt more easily. Since uniformity is defined in terms of a quality function, clustering would be the natural method to employ for this task. However, this method is only viable for frames with a large amount of annotation.</Paragraph>
    <Paragraph position="8"> A more general idea in this spirit is to construct an independent classification of verbs motivated at the argument structure level (transitive, intransitive, unaccusative, etc.), e.g. using data sources like Levin's verb classes (Levin, 1993). This would allow models to learn class-specific regularities and diathesis alternations more easily. However, it is unclear if there is a unique level at which all relevant regularities can be stated. A more realistic variant might be to map FrameNet roles to an existing, more syntactically oriented role set, such as PropBank. These roles can serve as an intermediate level to capture mapping regularities, and can be translated back to semantically defined FrameNet roles when the mapping has been accomplished.</Paragraph>
    <Paragraph position="9"> A third, different approach to semantic role assignment is presented by Frank (2004), who presents a syntax-semantics interface to extract symbolic frame element projection rules from an LFG-annotated corpus and discusses strategies to generalise over these rules. Such an approach is, due to the finer control over the generalisation, not as susceptible to the problem described in this study as purely statistical models. However, it has yet to be tested on large-scale semantic role assignment.</Paragraph>
  </Section>
class="xml-element"></Paper>