<?xml version="1.0" standalone="yes"?>
<Paper uid="N06-2018">
  <Title>MMR-based Active Machine Learning for Bio Named Entity Recognition</Title>
  <Section position="4" start_page="70" end_page="71" type="evalu">
    <SectionTitle>
3 Experiment and Discussion
</SectionTitle>
    <Paragraph position="0"/>
    <Section position="1" start_page="70" end_page="70" type="sub_section">
      <SectionTitle>
3.1 Experiment Setup
</SectionTitle>
      <Paragraph position="0"> We conducted our active learning experiments using pool-based sample selection (Lewis and Gale 1994). The pool-based sample selection, in which the learner chooses the best instances for labeling from a given pool of unlabelled examples, is the most practical approach for problems in which unlabelled data is relatively easily available.</Paragraph>
      <Paragraph position="1"> For our empirical evaluation of the active learning methods, we used the training and test data released by JNLPBA (Kim et al. 2004). The training corpus contains 2000 MEDLINE abstracts, and the test data contains 404 abstracts from the GENIA corpus. 100 abstracts were used to train our initial NER module. The remaining training data were taken as the pool. Each time, we chose k examples from the given pool to train the new NER module and the number k varied from 1000 to 17000 with a step size 1000.</Paragraph>
      <Paragraph position="2"> We test 4 different active learning methods: Random selection, Entropy-based uncertainty selection, Entropy combined with Diversity, and Normalized Entropy (equation (2)) combined with Diversity.</Paragraph>
      <Paragraph position="3"> When we compute the active learning score using the entropy based method and the combining methods we set the values of parameter N (from equation (1)) to 3 and l (from equation (3)) to 0.8 empirically.</Paragraph>
      <Paragraph position="4"> Fig1. Comparison of active learning strategies with the ranl in the y-axis shows the per bin ies consistently outperform the dom selection</Paragraph>
    </Section>
    <Section position="2" start_page="70" end_page="71" type="sub_section">
      <SectionTitle>
3.2 Results and Analyses
</SectionTitle>
      <Paragraph position="0"> The initial NER module gets an F-score of 52.54, while the F-score performance of the NER module using the whole training data set is 67.19. We plotted the learning curves for the different sample selection strategies. The interval in the x-axis between the curves shows the number of examples selected and the interva formance improved.</Paragraph>
      <Paragraph position="1"> We compared the entropy, entropy combined with sentence diversity, normalized entropy comed with sentence diversity and random selection. The curves in Figure 1 show the relative performance. The F-score increases along with the number of selected examples and receives the best performance when all the examples in the pool are selected. The results suggest that all three kinds of active learning strateg random selection.</Paragraph>
      <Paragraph position="2"> The entropy-based example selection has improved performance compared with the random selection. The entropy (N=3) curve approaches to the random selection around 13000 sentences selected, which is reasonable since all the methods choose the examples from the same given pool. As  the number of selected sentences approaches the pool size, the performance difference among the different methods gets small. The best performance of the entropy strategy is 67.31 when 17000 example null the normalized combined strategy behaves the worst.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>