<?xml version="1.0" standalone="yes"?>
<Paper uid="N04-4005">
  <Title>Enhancing Linguistically Oriented Automatic Keyword Extraction</Title>
  <Section position="6" start_page="0" end_page="0" type="concl">
    <SectionTitle>
5 Concluding Remarks
</SectionTitle>
    <Paragraph position="0"> In this paper, a number of experiments leading to a better performance of a keyword extraction algorithm has been presented. One improvement concerns how the NP-chunks are extracted, where the results are improved by excluding the initial determiners a, an, and the. Possibly, this improvement could be yet higher if all initial determiners were removed from the NP. Another improvement concerns how the collection frequency is calculated, where the F-measure of the extraction increases when a general corpus is used. A third improvement concerns how the weights to the positive examples are set. By adjusting the weights to maximise the performance of the combined model, the F-measure increases. Also, one major change is made to the algorithm, as the learning task is redefined. This is done by using regression instead of classification for the machine learning. Apart from an increase in performance by regression, this enables a ranked output of the keywords. This in turn makes it easy to vary the number of keywords selected per document, in case necessary for some types of applications. In addition, compared to classification, regression resembles reality in the sense that some words are definitely keywords, some are definitely not, but there are also many candidate terms that are keywords to a certain extent.</Paragraph>
    <Paragraph position="1"> Thus, there is a continuum of the candidate terms' &amp;quot;keywordness&amp;quot;. null Evaluating automatically extracted keywords is not trivial, as different persons may prefer different terms at different occasions. This is also true for professional indexers, where the consistency also depends on how experienced an indexer is. For example, Bureau van Dijk (1995) has shown that the index consistency between experienced indexers may be up to 60-80 per cent, while it is not unusual that it is as low as 20-30 between inexperienced indexers. The approach taken to the evaluation of the experiments presented in this paper is that of using keywords previously assigned by professional indexers as a gold standard for calculating the precision, the recall, and the F-measure. If looking at the inter-judgement agreement between the keywords selected by the combined model assigning no more than twelve keywords per document and the manually assigned keywords for the documents in the test set, it is 28.2%. Thus the performance of the keyword extraction algorithm is at least as consistent as that of inexperienced professional indexers.</Paragraph>
    <Paragraph position="2"> This is, however, only true to a certain extent, as some of the keywords selected by the automatic extractor would never have been considered by a human--not even a nonprofessional3. null</Paragraph>
  </Section>
class="xml-element"></Paper>