<?xml version="1.0" standalone="yes"?>
<Paper uid="C00-1030">
  <Title>Extracting the Names of Genes and Gene Products with a Hidden Markov Model</Title>
  <Section position="6" start_page="204" end_page="205" type="evalu">
    <SectionTitle>
4.2 Results
</SectionTitle>
    <Paragraph position="0"> The results are given as F-scores, a (;Ollllll()ll measurement for a(:(:ura(:y in tlw, MUC conferences that eonfl)ines r(;(:all and 1)re(:ision.</Paragraph>
    <Paragraph position="1"> These are eah:ulated using a standard MUC tool (Chinchor, 1995). F-score is d('.iin(~d as</Paragraph>
    <Paragraph position="3"> The tirst set ot7 experiments we did shows the effectiveness of the mode.1 for all name (:lasses and is smnmarized in Table 3. We see that data sparseness does have an etfe('t~ with 1)roteins the most mlmerous (;lass in training - getting the best result and I/,NA - the snmllc, st training (:lass - getting the worst result. The tal)le also shows the ett'eetiveness of the character feature set, whi('h in general adds 10.6% to the F-score.</Paragraph>
    <Paragraph position="4"> This is mainly due to a t)ositive effect on words in the 1)R,OTEIN and DNA elases, but we also see that memt)ers of all SOURCE sul)-('lasses sufl'er from featurization.</Paragraph>
    <Paragraph position="5"> We have atteml)ted to incorl)orate generalisation through character t'eatm:es and linear interi)olation, which has generally \])een quite su(:cessful. Nevertheless we were (:urious to see just  T(!xis indicates the mmfl)er of al)stra(:ts used ill training.</Paragraph>
    <Paragraph position="6"> whi(:h t)arts of the model were contributing to the bigram s(:ores. Table 4 shows the l)ercentage of bigranls which could be mat('hed against training t)igrams. The result indicate tha~ a high 1)ereentage of dire(:t bigrams in the test eorl)uS never al)t)(;ar in the training (:oft)us and shows tha, t our HMM model is highly depel&gt; (l(mt on smoothing through models ~kl and )~:~.</Paragraph>
    <Paragraph position="7"> \Y=e can take another view of the training data 1)y 'salalni-slieing' the model so that only evi(tenee from 1)art of the model is used. Results are shown in Tat)le 5 and support the eonchlsion that models Al, A2 and Aa are. crucial at this sir,(; of training data, although we would expect their relative ilnportance to fifil as we have more (tircct observations of bigrams with larger training data sets.</Paragraph>
    <Paragraph position="8"> Tal)le 6 shows the rolmstness of the model  stracts).</Paragraph>
    <Paragraph position="9"> F-score for all classes agMnst size of corpus (in number of MEDLINE abfor data sparseness, so that even with only 10 training texts the model can still make sensible decisions about term identification and classification. As we would expect;, the table ;flso clearly shows that more training data is better, and we have not yet reached a peak in pertbrinance. null</Paragraph>
  </Section>
class="xml-element"></Paper>