<?xml version="1.0" standalone="yes"?>
<Paper uid="W04-1215">
  <Title>Annotating Multiple Types of Biomedical Entities: A Single Word Classification Approach</Title>
  <Section position="4" start_page="80" end_page="80" type="relat">
    <SectionTitle>
3 Results and Discussion
</SectionTitle>
    <Paragraph position="0"> Since there is a huge amount of training instances and we do not have enough time to tune the parameters and train a model with all the training instances available, we first randomly selected one tenth and one fourth of the complete training data.</Paragraph>
    <Paragraph position="1"> The results, as we expected, showed that model trained with more instances performed better (see Table 2). However, we noticed that the performances vary among the 6 types and one of the possible causes is the imbalance of training data among classes (see Table 1). Therefore we decided to balance the training data.</Paragraph>
    <Paragraph position="2"> First, the training data was constructed to comprise equal number of instances from each class. However, it didn't perform well and lots of type 'O' words were misclassified, indicating that using only less than 1% of type 'O' training instances is not sufficient to train a good model.</Paragraph>
    <Paragraph position="3"> Thus two more models were trained to see if the performance can be enhanced. One model has slightly more type 'O' instances than the equally balanced one, and the other model has the ratio among classes being 4:8:4:1:8:16. The results showed increase in recall but drop in precision.</Paragraph>
    <Paragraph position="4"> Kazama et al. (2002) addressed the data imbalance problem and sped up the training process by splitting the type 'O' instances into sub-classes using part-of-speech information. However, we missed their work while we were doing this task, and hence didn't have the chance to use and extend this idea.</Paragraph>
    <Paragraph position="5"> After carefully examining the classification results, we found that many of the 'DNA' instances were classified as 'protein' and many of the 'protein' instances were classified as 'DNA'.</Paragraph>
    <Paragraph position="6"> For example, 904 out of 2,845 'DNA' instances were categorized as 'protein' under 'model 1/4'.</Paragraph>
    <Paragraph position="7"> The reason may be that Yapex and GAPSCORE do not distin guish gene name from protein names.</Paragraph>
    <Paragraph position="8"> Even humans don't do very well at this (Krauthammer et al., 2002).</Paragraph>
    <Paragraph position="9"> We originally planned to verify the contribution of each type of features. For example, how much noise was introduced by using existing taggers instead of lexicons. This would have helped gain more insights into the proposed features.</Paragraph>
  </Section>
class="xml-element"></Paper>