<?xml version="1.0" standalone="yes"?>
<Paper uid="W06-1616">
  <Title>Sydney, July 2006. c(c)2006 Association for Computational Linguistics Incremental Integer Linear Programming for Non-projective Dependency Parsing</Title>
  <Section position="3" start_page="0" end_page="129" type="intro">
    <SectionTitle>
1 Introduction
</SectionTitle>
    <Paragraph position="0"> Many inference algorithms require models to make strong assumptions of conditional independence between variables. For example, the Viterbi algorithm used for decoding in conditional random elds requires the model to be Markovian.</Paragraph>
    <Paragraph position="1"> Strong assumptions are also made in the case of McDonald et al.'s (2005b) non-projective dependency parsing model. Here attachment decisions are made independently of one another1. However, often such assumptions can not be justi ed. For example in dependency parsing, if a subject has already been identi ed for a given verb, then the probability of attaching a second subject to the verb is zero. Similarly, if we nd that one coordination argument is a noun, then the other argu1If we ignore the constraint that dependency trees must be cycle-free (see sections 2 and 3 for details).</Paragraph>
    <Paragraph position="2"> ment cannot be a verb. Thus decisions are often co-dependent.</Paragraph>
    <Paragraph position="3"> Integer Linear Programming (ILP) has recently been applied to inference in sequential conditional random elds (Roth and Yih, 2004), this has allowed the use of truly global constraints during inference. However, it is not possible to use this approach directly for a complex task like non-projective dependency parsing due to the exponential number of constraints required to prevent cycles occurring in the dependency graph.</Paragraph>
    <Paragraph position="4"> To model all these constraints explicitly would result in an ILP formulation too large to solve ef ciently (Williams, 2002). A similar problem also occurs in an ILP formulation for machine translation which treats decoding as the Travelling Salesman Problem (Germann et al., 2001).</Paragraph>
    <Paragraph position="5"> In this paper we present a method which extends the applicability of ILP to a more complex set of problems. Instead of adding all the constraints we wish to capture to the formulation, we rst solve the program with a fraction of the constraints. The solution is then examined and, if required, additional constraints are added. This procedure is repeated until all constraints are satis ed. We apply this dependency parsing approach to Dutch due to the language's non-projective nature, and take the parser of McDonald et al. (2005b) as a starting point for our model.</Paragraph>
    <Paragraph position="6"> In the following section we introduce dependency parsing and review previous work. In Section 3 we present our model and formulate it as an ILP problem with a set of linguistically motivated constraints. We include details of an incremental algorithm used to solve this formulation. Our experimental set-up is provided in Section 4 and is followed by results in Section 5 along with runtime experiments. We nally discuss fu-</Paragraph>
  </Section>
class="xml-element"></Paper>