<?xml version="1.0" standalone="yes"?>
<Paper uid="W04-0814">
  <Title>The University of Amsterdam at Senseval-3: Semantic Roles and Logic Forms</Title>
  <Section position="2" start_page="0" end_page="0" type="intro">
    <SectionTitle>
1 Introduction
</SectionTitle>
    <Paragraph position="0"> This year (2004), Senseval, a well-established forum for the evaluation and comparison of word sense disambiguation (WSD) systems, introduced two tasks aimed at building semantic representations of natural language sentences. One task, Automatic Labeling of Semantic Roles (SR), takes as its theoretical foundation Frame Semantics (Fillmore, 1977) and uses FrameNet (Johnson et al., 2003) as a data resource for evaluation and system development. The definition of the task is simple: given a natural language sentence and a target word in the sentence, find other fragments (continuous word sequences) of the sentence that correspond to elements of the semantic frame, that is, that serve as arguments of the predicate introduced by the target word.</Paragraph>
    <Paragraph position="1"> For this task, the systems receive a sentence, a target word, and a semantic frame (one target word may belong to multiple frames; hence, for real-world applications, a preliminary WSD step might be needed to select an appropriate frame). The output of a system is a list of frame elements, with their names and character positions in the sentence. The evaluation of the SR task is based on precision and recall. For this year's task, the organizers chose 40 frames from FrameNet 1.1, with 32,560 annnotated sentences, 8,002 of which formed the test set.</Paragraph>
    <Paragraph position="2"> The second task, Identification of Logic Forms in English (LF), is based on the LF formalism described in (Rus, 2002). The LF formalism is a simple logical form language for natural language semantics with only predicates and variables; there is no quantification or negation, and atomic predications are implicitly conjoined. Predicates correspond directly to words and are composed of the base form of the word, the part of speech tag, and a sense number (corresponding to the WordNet sense of the word as used). For the task, the system is given sentences and must produce LFs. Word sense disambiguation is not part of the task, so the predicates need not specify WordNet senses. System evaluation is based on precision and recall of predicates and predicates together with all their arguments as compared to a gold standard.</Paragraph>
  </Section>
class="xml-element"></Paper>