<?xml version="1.0" standalone="yes"?>
<Paper uid="W04-3101">
  <Title>A resource for constructing customized test suites for molecular biology entity identification systems</Title>
  <Section position="6" start_page="0" end_page="0" type="concl">
    <SectionTitle>
4 Conclusion
</SectionTitle>
    <Paragraph position="0"> We do not advocate using this approach to replace the quantitative evaluation of EI systems by precision, recall, and F-measure. Arguably, overall performance on real corpora is the best evaluation metric for entity identification, in which case the standard metrics are well-suited to the task. However, at specific points in the software lifecycle, viz. during development and at the time of acceptance testing, the standard metrics do not provide the right kind of information. We can, however, get at this information if we bear in mind two things: 1. Entity identification systems are software, and as such can be assessed by standard software testing techniques.</Paragraph>
    <Paragraph position="1"> 2. Entity identification systems are in some sense instantiations of hypotheses about linguistic structure, and as such can be assessed by standard linguistic &amp;quot;field methods.&amp;quot; This paper describes a methodology and a data set for utilizing the principles of software engineering and linguistic analysis to generate test suites that answer the right kinds of questions for developers and for end users. Readers are invited to contribute their own data.</Paragraph>
  </Section>
class="xml-element"></Paper>