<?xml version="1.0" standalone="yes"?>
<Paper uid="M98-1005">
  <Title>UNIVERSITY OF DURHAM: DESCRIPTION OF THE LOLITA SYSTEM AS USED IN MUC-7</Title>
  <Section position="9" start_page="0" end_page="0" type="concl">
    <SectionTitle>
CONCLUSIONS
</SectionTitle>
    <Paragraph position="0"> The LOLITA system is a natural language processor that has a core functionality on which a number of different applications can be built. The system was not designed to compete solely in the MUC tasks and in fact the MUC specific code forms a very small part of the whole. The first MUC evaluation to which LOLITA was applied was MUC-6. Since then the system's core has been substantially improved and this has lead to a significant improvement in the system's performance for the MUC-7 evaluation. We are pleased with this improvement and believe that this further validates our method of development.</Paragraph>
    <Paragraph position="1"> Although pleased with the results there is certainly room for improvement. In particular approximately 20% of parses failed in the final evaluation. This had a substantial impact on the system's performance as all of the task applications rely on a `good' core analysis of which parsing is a key component. Although some recovery strategies were used, these had a limited effect. (Little effort was expanded in attempting to reduce the number of parse failures as a new parser is under development.) Of particular encouragement were the articles for which the system's underlying analysis was correct. In such cases the system was able to apply the higher level semantic and pragmatic rules to good effect. The system also suffered a drop in scores due to the lack of data that was available.</Paragraph>
    <Paragraph position="2"> The issues of more robust parsing and data improvement are currently being addressed. Major projects are underway to re-engineer large parts of the system (the parser being one), and to add a large amount of data in the form of dictionary definitions.</Paragraph>
    <Paragraph position="3"> We feel that the MUC series of evaluations has been very useful to the development of the LOLITA system and has allowed us to focus development on some key areas of the system's core. Now that MUC has reached an end we shall continue to devote our main development effort to the core issues, in particular those mentioned above, i.e., re-engineering of the system, and addition of dictionary definition data. This engineering work, carried out by a university spin-off company, 3F Ltd, will allow us to reach our goal of bringing to the market, a number of sophisticated and powerful natural language processing applications based on an underlying core system.</Paragraph>
  </Section>
class="xml-element"></Paper>