<?xml version="1.0" standalone="yes"?>
<Paper uid="W05-1201">
  <Title>Classification of semantic relations by humans and machines [?] Erwin Marsi and Emiel Krahmer Communication and Cognition</Title>
  <Section position="7" start_page="5" end_page="5" type="concl">
    <SectionTitle>
5 Discussion and Future work
</SectionTitle>
    <Paragraph position="0"> This paper presented an approach to detecting semantic relations at the word, phrase and sentence level on the basis of dependency analyses. We investigated the performance of human annotators on the tasks of manually aligning dependency analyses and of labeling the semantic relations between aligned nodes. Results indicate that humans can perform this task well, with an F-score of .98 on alignment and an F-score of .92 on semantic relations (after revision).</Paragraph>
    <Paragraph position="1"> We also described and evaluated automatic methods addressing these tasks: a dynamic programming tree alignment algorithm which achieved an F-score on alignment of .85 (using lexical semantic information from EuroWordNet), and a memory-based semantic relation classifier which achieved F-scores of .64 and .80 with and without using real previous decisions respectively.</Paragraph>
    <Paragraph position="2"> One of the issues that remains to be addressed in future work is the effect of parsing errors. Such errors were not corrected, but during manual alignment, we sometimes found that substrings could not be properly aligned because the parser had failed to identify them as syntactic constituents. As far as classification of semantic relations is concerned, the proliferation of classification errors is an issue that needs to be solved. Classification performance may be further improved with additional features (e.g.</Paragraph>
    <Paragraph position="3"> phrase length information), optimization, and more data. Also, we have not yet tried to combine automatic alignment and classification. Yet another point concerns the type of text material. The sentence pairs from our current corpus are relatively close, in the sense that both translations more or less convey the same information. Although this seems a good starting point to study alignment, we intend to continue with other types of text material in future work. For instance, in extending our work to the actual output of a QA system, we expect to encounter sentences with far less overlap.</Paragraph>
  </Section>
class="xml-element"></Paper>