<?xml version="1.0" standalone="yes"?>
<Paper uid="W99-0608">
  <Title>Improving POS Tagging Using Machine-Learning Techniques</Title>
  <Section position="4" start_page="0" end_page="53" type="intro">
    <SectionTitle>
1 Introduction
</SectionTitle>
    <Paragraph position="0"> The study of general methods to improve the performance in classification tasks, by the combination of different individual classifiers, is a currently very active area of research in supervised learning. In the machine learning (ML) literature this approach is known as ensemble, stacked, or combined classifiers. Given a classification problem, the main goal is to construct several independent classifiers, since it has been proven that when the errors committed by individual classifiers are uncorrelated to a sufficient degree, and their error rates are low enough, the resulting combined classifier performs better than all the individual systems (Ali and Pazzani, 1996; Tumer and Ghosh, 1996; Dietterich, 1997).</Paragraph>
    <Paragraph position="1"> Several methods have been proposed in order to construct ensembles of classifiers that make uncorrelated errors. Some of them are general, and they can be applied to any learning algorithln, while other are specific to particular algorithms. From a different perspective, there exist methods for constructing homogeneous ensembles, in the sense that a unique learning algorithm has been used to acquire each individual classifier, and heterogeneous ensembles that combine different types of learning paradigms 1.</Paragraph>
    <Paragraph position="2"> Impressive results have been obtained by applying these techniques on the so-called unstable learning algorithms (e.g. induction of decision trees, neural networks, rule-induction systems, etc.). Several applications to real tasks have been performed, and, regarding NLP, we find ensembles of classifiers in context-sensitive spelling correction (Golding and Roth, 1999), text categorization (Schapire and Singer, 1998; Blum and Mitchell, 1998), and text filtering (Schapire et al., 1998). Combination of classitiers have also been applied to POS tagging. For instance, van Halteren (1996) combined a number of similar tuggers by way of a straightforward majority vote. More recently, two parallel works (van Halteren et al., 1998; Brill and Wu, 1998) combined, with a remarkable success, the output of a set of four tuggers based on different principles and feature modelling. Finally, in the work by MSxquez et al. (1998) the combination of taggers is used in a bootstrapping algorithm to train a part of speech tagger from a limited amount of training material.</Paragraph>
    <Paragraph position="3"> The aim of the present work is to improve an existing POS tagger based on decision trees (Mkrquez and Rodriguez, 1997), by using ensembles of classifiers. This tagger treats separately the different types (classes) of ambiguity by considering a different decision tree for each class. This fact allows a selective construction of ensembles of decision trees focusing on the most relevant ambiguity classes, which greatly vary in size and difficulty. Another goal of the present work is to try to alleviate the problem of data sparseness by applying a method, due  to Breiman (1998), for generating new pseudo-examples from existing data. As we will see in section 4.2 this technique will be combined with the construction of an ensemble of classifiers.</Paragraph>
    <Paragraph position="4"> The paper is organized as follows: we start by presenting the two versions of the POS tagger and their evaluation on the reference corpus (sections 2 and 3). Sections 4 and 5 are, respectively, devoted to present the machine-learning improvements and to test their implementation.</Paragraph>
    <Paragraph position="5"> Finally, section 6 concludes.</Paragraph>
  </Section>
class="xml-element"></Paper>