<?xml version="1.0" standalone="yes"?>
<Paper uid="W02-2018">
  <Title>A comparison of algorithms for maximum entropy parameter estimation</Title>
  <Section position="4" start_page="0" end_page="0" type="concl">
    <SectionTitle>
4 Conclusions
</SectionTitle>
    <Paragraph position="0"> In this paper, we have described experiments comparing the performance of a number of different algorithms for estimating the parameters of a conditional ME model. The results show that variants of iterative scaling, the algorithms which are most widely used in the literature, perform quite poorly when compared to general function optimization algorithms such as conjugate gradient and variable metric methods. And, more specifically, for the NLP classification tasks considered, the limited memory variable metric algorithm of Benson and Mor'e (2001) outperforms the other choices by a substantial margin.</Paragraph>
    <Paragraph position="1"> This conclusion has obvious consequences for the field. ME modeling is a commonly used machine learning technique, and the application of improved parameter estimation algorithms will it practical to construct larger, more complex models. And, since the parameters of individual models can be estimated quite quickly, this will further open up the possibility for more sophisticated model and feature selection techniques which compare large numbers of alternative model specifications. This suggests that more comprehensive experiments to compare the convergence rate and accuracy of various algorithms on a wider range of problems is called for.</Paragraph>
    <Paragraph position="2"> In addition, there is a larger lesson to be drawn from these results. We typically think of computational linguistics as being primarily a symbolic discipline. However, statistical natural language processing involves non-trivial numeric computations.</Paragraph>
    <Paragraph position="3"> As these results show, natural language processing can take great advantage of the algorithms and software libraries developed by and for more quantitatively oriented engineering and computational sciences. null</Paragraph>
  </Section>
class="xml-element"></Paper>