<?xml version="1.0" standalone="yes"?>
<Paper uid="W99-0407">
  <Title>FAME: a Functional Annotation Meta-scheme for multi-modal and multi-lingual Parsing Evaluation</Title>
  <Section position="4" start_page="45" end_page="45" type="concl">
    <SectionTitle>
4 Conclusion and developments
</SectionTitle>
    <Paragraph position="0"> The suggestion of using a functional meta-scheme as a fair basis for parsing evaluation rests on the idea that parsing systems must be assessed for what they are intended to provide, not for how well they meet the requisites of other annotation schemes. Still; it makes a lot of sense to compare the amount of information provided by different parsers by casting this information into a common format. The distributed information structure of FAME is conducive to an incremental evaluation procedure, which ranges from a base evaluation level (involving sheer identification of the terms of a syntactic relationship and/or their order), to more refined levels, including morpho-syntactic information, dependency type, and ultimately predicate--argument structure. The evaluation of a text annotated for functional information can then be conceived of as a function of estimating precision and recall for each of the independent evaluation levels envisaged. Evaluation results obtained for the different levels can eventually be combined together or, for particular purposes, assessed in their own right (e.g. for IR applications the basic evaluation level could be sufficient). We are considering the possibility of extending FAME through addition of still further levels of lingustic information.</Paragraph>
  </Section>
class="xml-element"></Paper>