<?xml version="1.0" standalone="yes"?>
<Paper uid="W04-0810">
  <Title>A First Evaluation of Logic Form Identification Systems</Title>
  <Section position="10" start_page="0" end_page="0" type="concl">
    <SectionTitle>
9 Conclusion
</SectionTitle>
    <Paragraph position="0"> In the first attempt to systematically evaluate LFi systems we managed to provide a gold standard, a first software evaluator and proposed few performance metrics that can be used by researchers in the community to further study their approaches.</Paragraph>
    <Paragraph position="1"> Among the drawbacks, it is worth mentioning the lack of training data which we plan to offer in the future.</Paragraph>
    <Paragraph position="2"> The results reported by different systems constitute a lower bound of their approaches since the test data comprised raw sentences and thus the reported performances include errors coming from tokenization, part of speech tagging and parsing, wherever parsing was used.</Paragraph>
    <Paragraph position="3"> Due to a tight schedule it was not possible to analyze the different approaches adopted by different systems but we hope the ACL meeting will provide the necessary background information and discussions to foster the development of such a study.</Paragraph>
  </Section>
class="xml-element"></Paper>