<?xml version="1.0" standalone="yes"?>
<Paper uid="W00-1109">
  <Title>From Information Retrieval to Information Extraction</Title>
  <Section position="7" start_page="92" end_page="92" type="evalu">
    <SectionTitle>
5 Evaluation
</SectionTitle>
    <Paragraph position="0"> We have not yet performed a large-scale evaluation of the user-customisable version of Highlight.</Paragraph>
    <Paragraph position="1"> The earlier version of Highlight was evaluated for the task of extracting protein interactions, obtaining a recall of 55-58% and precision of 77%. For this particular task we would not expect the results for the new system to differ significantly: the underlying engine is the same and we can use the interface to generate sets of constraints which are equivalent to the previous patterns. For example, Figures 5 (above) and 9 show how an old pattern can be generated in the new interface. Both look for phrases such as ... observed that TAF binds TBP, the old pattern places an interaction tag around the relevant text while in the new interface the user selects which items will be output (in this case, the ones which have names rather than numbers.) For other tasks, there are some factors which might affect the relative recall and precision figures between the original Highlight system and the customisable version. Firstly, the original Highlight system can use cascades of pattern matching rules to perform 'blocking tactics' i.e.</Paragraph>
    <Paragraph position="2"> one rule fires for the sole purpose of preventing another from firing. To achieve a similar ability in customisable Highlight we need to introduce a way of choosing between alternative queries when more than one can apply e.g. by always choosing the most specific. Secondly, the original Highlight sometimes resorted to external program calls which is no longer possible.</Paragraph>
  </Section>
class="xml-element"></Paper>