<?xml version="1.0" standalone="yes"?>
<Paper uid="N04-2009">
  <Title>Construction of Conceptual Graph representation of texts</Title>
  <Section position="6" start_page="0" end_page="0" type="evalu">
    <SectionTitle>
5 Experimental results
</SectionTitle>
    <Paragraph position="0"> We currently are in the process of testing and tuning our system. We have some preliminary results for the performance of the semantic role annotation module, both on Reuters news articles and AAIU reports. The tests on the Reuters documents are performed on a quarter of the available corpus (reut2-003.sgm) and for the AAIU documents on the reports from years 1998, 1999 and 2000.</Paragraph>
    <Paragraph position="1"> The coverage (the percentage of the verbs in the corpus that have a VerbNet description) of VerbNet for both corpora is relatively low: 66% for Reuters and 53% for the AAIU.</Paragraph>
    <Paragraph position="2"> To evaluate the performance of the semantic role labelling algorithm we randomly selected 1% of the verbs from each corpus and manually analysed the assigned semantic roles. Our tests show that the semantic roles are correctly identified in 39% of cases in Reuters corpus and 35% of the cases in the AAIU reports, which is 59% and 66% respectively of the verbs present in VerbNet (the percentage of the correctly identified out of all that are covered by VerbNet).</Paragraph>
    <Paragraph position="3"> We are currently extending the coverage of VerbNet by manually identifying frames present in the corpora and not included in VerbNet, which we believe should significantly increase the performance.</Paragraph>
  </Section>
class="xml-element"></Paper>