<?xml version="1.0" standalone="yes"?>
<Paper uid="C02-1088">
  <Title>Unsupervised Named Entity Classification Models and their Ensembles</Title>
  <Section position="5" start_page="3" end_page="3" type="evalu">
    <SectionTitle>
3 Experimental Results
</SectionTitle>
    <Paragraph position="0"> We used Korean news articles that consist of 24,647 eojeols and contain 2,580 named entities as a test set. The number of named entities which belong to each category is shown in Table 2. When even a human could not classify named entities, 'Unknown' is labeled and it is ignored for the evaluation. 'Other' is used for the word outside the three categories.</Paragraph>
    <Paragraph position="1"> Table 3 shows the result of the classification.</Paragraph>
    <Paragraph position="2"> The first row shows the result of the classification using only a NE dictionary. The recall (14.84%) is very low because the system uses a small-scale dictionary. The precision (91.56%) is not 100% because of the semantic ambiguity. It means that it is necessary to refine classifications created by a dictionary.</Paragraph>
    <Paragraph position="3"> We build a training set with a NE dictionary and a POS tagged corpus and refine it with co-occurrence information. The second row shows the result of the classification using this training set without learning. We can observe that the quality of the training set is improved thanks to our refining method.</Paragraph>
    <Paragraph position="4"> A Mixed Voting shows the best results. It improves the performance by taking good characteristics of a majority voting and probability voting.</Paragraph>
    <Paragraph position="5">  We extract the syntatic relations and make 5 windows (modifier, target word, modifiee, josa, predicate) as a context. We conduct a comparative experiment using the Uchimoto's method, 5 windows (two words before/after the target word) and then we show that our method brings to a better result (Table 4).</Paragraph>
    <Paragraph position="6">  We try to perform the co-training similar to one of Colins and Singer in the same experimental environment. We extract contextual rules from our 5 windows because we does not have a full parser. The learning is started from 417 spelling seed rules made by the NE dictionary. We use two independent context and spelling rules in turn. Table 5 shows that our method improve the recall much more on the same conditions.</Paragraph>
    <Paragraph position="7">  Through the ensemble of various learning methods, we get larger and more precise training examples for the classification. Table 6 shows that the ensemble learning brings a better result  Three learners can use different kinds of features instead of same features. We conduct a comparative experiment as following. As features, SNoW uses a modifier and a target word, Timbl uses a modifiee and a target word, and MEMT uses a josa, a predicate and a target word. Table 7 shows that the learning using different kinds of features has the low performance because of the lack of information.  The system repeats the learning with new training examples generated through the ensemble learning. We can see that this loop brings to the better result as shown in Table 8. After the learning, we apply the rule, a sense per discourse. 'Post' in Table 8 indicates the performance after this post-processing. It The post-processing improves the performance a little.</Paragraph>
    <Paragraph position="8">  We extracted the syntactic relations by using a simple heuristic parser. Because this parser does not deal with complex sentences, the failure of parsing causes the lack of information or wrong learning. Most of errors are actually occurred by it, therefore we need to improve the performance of the parser.</Paragraph>
  </Section>
class="xml-element"></Paper>