<?xml version="1.0" standalone="yes"?>
<Paper uid="H94-1073">
  <Title>Assessing the Retrieval Effectiveness of a Speech :Retrieval System by Simulating Recognition Errors</Title>
  <Section position="4" start_page="371" end_page="371" type="evalu">
    <SectionTitle>
3. Results
</SectionTitle>
    <Paragraph position="0"> Table 1 shows the average precision values for various detection rates and false alarm rates. The numbers in brackets represent the percentage of a average precision of the reference method. The reference method represents a standard text retrieval method which is based on words rather than on VCV-features. In our case here, the reference method uses van Rijsbergen's \[13\] stoplist to eliminate the high-frequency words. Furthermore, it uses the word reduction algorithm by Porter \[9\] to reduce different variants of a words to the same normal form. The term weights consist of simple tf * idf weights and the retrieval status values are obtained by the cosine measure.</Paragraph>
    <Paragraph position="1"> Current wordspotting systems report high detection rates and low false alarms for the recognition of entire words in speech documents \[2\],\[3\],\[4\]. These systems, however, are usually evaluated on small tasks: the vocabulary of the speech database is in the order of 1000 words and the number of words spotted is small. On the other hand, the task to identify 1000 VCV-features in speech documents with an unlimited vocabulary is much more difficult and the corresponding false alarms are one to two orders of magnitude higher. We therefore consider detection rates within the range of 40% and 90% and false alarms per keyword (i.e. per indexing fea~ ture) per hour within the range of 0 and 140.</Paragraph>
  </Section>
class="xml-element"></Paper>