<?xml version="1.0" standalone="yes"?>
<Paper uid="W04-1504">
  <Title>Axiomatization of Restricted Non-Projective Dependency Trees through Finite-State Constraints that Analyse Crossing Bracketings</Title>
  <Section position="4" start_page="0" end_page="0" type="metho">
    <SectionTitle>
3 Proper Embracement Depth
</SectionTitle>
    <Paragraph position="0"> In the context of D-trees, a counterpart notion for center-embedding of constituent trees is needed. We say that an arc a2 a3 a5 a2a13a3 properly embraces another arc a2 a7 a5 a2a8a7 , if and only if a3 a4a6a5 a19 a2 a3 a37 a2a13a3a75a24a8a7</Paragraph>
    <Paragraph position="2"> where a7 is the linear precedence order among the nodes. The proper embracement depth a0 of a colored D-tree is the maximum number a0 of arcs a2 a3 a5 a2a13a3 , a2 a7 a5 a2 a7 , a74a75a74a75a74 , a2a16a15 a5 a2 a15 where all the arcs have the same color and each a2 a12 a5 a2 a12, a0 a13 a17a22a13</Paragraph>
    <Paragraph position="4"> sure is applied to a D-tree in the Figure 3. Note that proper embracement does not generally imply that the arcs belong to a common path: in Figure  clause is 3.</Paragraph>
    <Paragraph position="5"> An arc a2 a3a6a5 a2a13a3 that shares a node with another arc a2 a7 a5 a2a8a7 does not properly embrace the later. If they overlap each other, the shorter of these arcs will be represented with a pair of an angle bracket and a square bracket as shown in Figure 2, unless the longer arc is already presented by an angle bracket. Thus, the maximum number of nested square brackets needed corresponds to the proper embracement depth of the colored D-tree.</Paragraph>
    <Paragraph position="6"> In our representation, the proper embracement depth of trees is bounded by a fixed parameter a0 . This allows defining finite-state constraints that define bracketings up to a bounded number of nested square brackets. In the following, we will give axioms that check that brackets for each color are balanced and do not exceed the proper embracement depth a0 .</Paragraph>
    <Paragraph position="7"> We define first, for each color a17 a36a8a18a0 a37 a1 a37 a2 a37a75a74a75a74a75a74 a37 a1a20a19 , a string homomorphism a21 a12 a18 a46a64a48a23a22 a19 [a12a37 ]a12a24a25a48 in such a way that it essentially deletes all the other symbols except the brackets [a12 and ]a12, and the</Paragraph>
    <Paragraph position="9"> Axiom 7. The [a12a37 ]a12-bracketings must be balanced and the number of nested brackets is bounded.</Paragraph>
    <Paragraph position="10"> Axiom 8. (a+b) Left (right) angle brackets match with a square bracket. (c) The arcs indicated with angle-square bracket pairs do not cross each other as in [a12a74a75a74a75a74a12 a36a12 a30a32a30a32a30a12 a34a12 a30a32a30a32a30a12 ]a12. These axioms are given more formally as follows:</Paragraph>
    <Paragraph position="12"/>
  </Section>
  <Section position="5" start_page="0" end_page="0" type="metho">
    <SectionTitle>
4 Nested Crossing Depth
</SectionTitle>
    <Paragraph position="0"> We added colors to brackets because crossing brackets have to be separated by some means.</Paragraph>
    <Paragraph position="1"> Unfortunately, assigning colors to brackets entails  new problems: 1. We can represent non-projective trees that are  not typical for natural language. In particular, we conjecture that although we bound the number of colors a1 available, there is a set of colored trees (in the limit a0 a22 a42 ) that gives structural descriptions for the Bach language (cf. Joshi 1985), the strings of which consist of an equal number of a's, b's and c's.</Paragraph>
    <Paragraph position="2"> 2. In parsing, the colors must be selected in one way or in another and this results normally into an ambiguity where there are many colorings available. Thus, we need a discipline that tells how to assign colors to the arcs in an unambiguous way.</Paragraph>
    <Paragraph position="3"> The first problem could be addressed e.g. by combining a constituent-based structure (topological fields etc.) with the dependency syntax. In our representation for D-trees, we need however a solution that addresses both of these problems. Such a solution has been developed recently and presented in many ways: by means of constraints (Yli-Jyr&amp;quot;a, 2003a), through an informal algorithm (Yli-Jyr&amp;quot;a, 2004b), and very formally as a special index storage type used in Colored Non-projective Dependency Grammar (Yli-Jyr&amp;quot;a and Nyk&amp;quot;anen, 2004). We conjecture, however, that there is no essential differences in the allocation disciplines defined in these works. In the following, we will adapt the constraint-based definition (Yli-Jyr&amp;quot;a, 2003a) to the allocation of colors of brackets: Axiom 9 (Plane locking). If a bracket [a0 is still open at a string position, the position cannot contain a bracket [a12 for which a17a2a1a4a3 .</Paragraph>
    <Paragraph position="4"> An effect of this axiom can be seen in Figure 2, where a new colora2 is selected after venit although the color a0 is no more in use. The reason is that there is a bracket [a7 that is still open at that position. Axiom 10 (Left conjoin). All the opening brackets belonging to the same node have the same color.</Paragraph>
    <Paragraph position="5"> This axiom corresponds intuitively to the fact that there is no need to give different colors to arcs that do not cross each other.</Paragraph>
    <Paragraph position="6"> Axiom 11 (Continuous tiling). A position cannot contain colored bracket [a0 , where a0a5a1a6a3a7a1 a1 , if, on the left, there are no other brackets [a0 (of the same color) that remain opened at the position, except if there is, on the left, another bracket [a12 with a8a10a9a12a11a14a13a5a15 (of the preceding color) that remains open at this position but will be matched with with a bracket a34a12 or ]a12 that occurs before the bracket [a0 of the current position is closed with ]a0 .</Paragraph>
    <Paragraph position="7"> This axiom corresponds to the fact that when a new color is introduced at some position (Figure 2), this is done due to a danger of having crossing brackets with the same color.</Paragraph>
    <Paragraph position="8"> The actual effect of these three axioms is that for each D-tree there remains a unique way to assign colors, square brackets, and angle brackets to the arcs. The nested crossing depth a1 of a D-tree is the number of colors in a colored D-tree that conforms Axioms 9 - 11. In our representation, the nested crossing depth (i.e. the number of colors) is bounded.</Paragraph>
    <Paragraph position="9"> Bounded nested crossing depth has considerable linguistic relevance. The length of the longest chain (a16a17a16a17a16 a30a32a30a32a30 a16a17a16 ) of crossing edges is a lower bound for the nested crossing depth, but such chains are typically very short in natural language sentences.</Paragraph>
    <Paragraph position="10"> The possible upper bound for the nested crossing depth has been studied experimentally (Yli-Jyr&amp;quot;a, 2003a; Yli-Jyr&amp;quot;a, 2004b) with the result that in non-projective D-trees of some 700 Danish sentences, the number of required colors is pretty low (a1 -a2 ). A few interesting exceptions1 actually contained a chain of up to five crossing dependencies. Such complex examples seem to be successful combinations of non-local dependencies, and it may be very difficult to generalize what is possible and what is not. Nevertheless, we conjecture that D-trees of the Bach (or MIX) language (cf. Joshi 1985) are not captured in our system when the number colors is fixed, because Colored Non-Projective Dependency Grammar (Yli-Jyr&amp;quot;a and Nyk&amp;quot;anen, 2004) is a linear context-free rewriting system.</Paragraph>
    <Paragraph position="11"> In order to facilitate formalization of Axiom 11, we use of color selectors and the following axiom: Axiom 12. (a) Color selector a18a3a13a18 , where a0a18a1a19a3 a13 a1 , indicates that there is a left bracket [a0 that has not yet been closed. (b) Color selector a3 a18 indicates that no bracket [a0 is open at that position.</Paragraph>
    <Paragraph position="12"> If we assume also a bound for the proper embracement depth, we can present the above axioms more formally as follows:</Paragraph>
    <Paragraph position="14"/>
  </Section>
  <Section position="6" start_page="0" end_page="0" type="metho">
    <SectionTitle>
5 Subcategorization
</SectionTitle>
    <Paragraph position="0"> We have seen in Section 3 that angle brackets are used when several overlapping arcs share a common node. This corresponds to use of reduced bracketing for initial and final embedding in some systems (Krauwer and des Tombe, 1981; Yli-Jyr&amp;quot;a, 2003b), and it facilitates linguistically appropriate bracketing with FSMs.</Paragraph>
    <Paragraph position="1"> Our axiomatization (Section 6 in particular) requires that information about the labels and directions on the arcs of the node are locally present both in the dependent and the governor nodes. Unfortunately, this kind of duplication of the labeling information cannot be captured with regular axioms unless there is a limit on the amount of information that is duplicated. A solution would be to assign each square bracket an unsaturated subcategorization frame with a symbol that indicates a state in a special subcategorization automaton. The automata could be simulated by propagating -- by means of 1e.g. &amp;quot;Det har</Paragraph>
    <Paragraph position="3"> declarative constraints -- the state information of each square bracket to the first angle bracket, and then further from one angle bracket to another. We have chosen, however, a more restricted approach for brevity, although we do not argue that it is the  square bracket correspond to the labels attached to angle brackets.</Paragraph>
    <Paragraph position="4"> We assume that the number of left or right arcs per color is bounded by an integer a0 . Thus, at most a0 labels can be associated with one square bracket. The label that is nearest to the opening (closing) square bracket corresponds to the label that is nearest to the corresponding closing (opening) square bracket. Each additional label of the square bracket corresponds to a label of an angle bracket (Figure 4).</Paragraph>
    <Paragraph position="5"> We will now give axioms that check that the labels of square brackets corresponds to the labels of the matching square and angle brackets: Axiom 13. (a)+(b) The number of left (right) angle brackets matching each right (left) square bracket is determined by the number of labels associated with the right (left) square bracket.</Paragraph>
    <Paragraph position="6"> Axiom 14. (a) Every label of the right square brackets has a corresponding bracket with a corresponding label. (b) Every label of the left square brackets has a corresponding bracket with a corresponding label.</Paragraph>
    <Paragraph position="7"> These axiom are formulated as follows:  and a8 a29 a25a5 and a0 a29 a25a5 describe what is inside the [a29a13a37 ]a29 square brackets, when the a31a14a11a16a15a35a0a33 th label of the left and right square bracket, respectively, has a matching label. These languages are defined as  is a language whose strings contain just matching pairs of labels and everything that can come between them.</Paragraph>
  </Section>
  <Section position="7" start_page="0" end_page="0" type="metho">
    <SectionTitle>
6 Non-Projectivity Depth
</SectionTitle>
    <Paragraph position="0"> The arcs in dependency trees constitute, by the definition of trees, an acyclic graph -- our discussion assumes that there are no secondary links in D-trees.</Paragraph>
    <Paragraph position="1"> In the axiomatization of the string representation, we have to enforce acyclicity by some constraints.</Paragraph>
    <Paragraph position="2"> Procedurally the acyclicity could be decided, for example, by trying to arrange the nodes into an order where the arcs go from the left to the right (topological sorting). Corresponding declarative solutions would be e.g. (i) to use set constraints (Duchier, 1999) or (ii) to attach each node an integer that increases strictly in the nodes reached by the outgoing arcs of the node. Both of these solutions are problematic because the number of reached nodes is, in practice, unbounded. An alternative solution that is adopted her is to use a monotonically increasing counter that is incremented only at certain critical positions. For technical reasons, we attach such a counter to arcs and brackets rather than to the nodes -- this change is not mathematically significant.</Paragraph>
    <Paragraph position="3"> Let a7 be the linear precedence relation over the nodes. A node in a D-tree is an articulation node if no arcs are passing it and the arcs coming into it are on the opposite side than the arcs going out from it. A chain of colored arcs a2 a3a31a5  if either (i) a2 a12a6a17 a3 a7 a2 a12a34 a3 a7 a2 a12, (ii) a2 a12 a7 a2 a12a34 a3 a7 a2 a12a6a17 a3 , or (iii) a1a12a34 a3 a37 a1a12a17 a3 . The non-projectivity depth of a (colored dependency) path that does not contain an articulation node is the number of critical nodes visited by it. The maximum depth of such paths in a D-tree is the non-projectivity depth of the D-tree.</Paragraph>
    <Paragraph position="4"> Incrementing counters only at the critical positions has an important advantage over the other solutions mentioned: projective trees do not contain any critical positions, and in non-projective trees of natural language sentences we probably need only very small numbers. If the counter is incremented several times, the path can be very &amp;quot;unnatural&amp;quot; as shown in Figure 5. We conjecture that if a de- null pendency path is acyclic, we cannot increment the counters at every critical position. In other words, assigning depths to the arcs of a D-graph excludes the alternative that the graph would be cyclic.</Paragraph>
    <Paragraph position="5"> In our representation, there is a fixed upper bound a2 for the non-projectivity depth. Based on it, we can define, for all a17a85a17 a18a38a10a37 a0 a37 a1 a37a75a74a75a74a75a74 a37 a2 a19 , the sets a26 a27 a25a12 a17</Paragraph>
    <Paragraph position="7"> the non-projectivity depth counter. The incrementation of these counters in critical nodes gives rise to the following axioms. They constrain nodes i.e.</Paragraph>
    <Paragraph position="8"> substrings occurring between two word boundaries: Axiom 15. (a) In nodes that are not articulation nodes, there is no label a39a44a36 a26 a27 a25a12, wherea38 a13 a17 a1 a2 , if there is a labela77 a36 a26a28a42 a25a0 , a17 a1 a3 a13 a2 . (b) There is no label a39a60a36 a26 a27 a25a0 , where a38 a1 a3 a13 a2 , if there is no</Paragraph>
    <Paragraph position="10"> to closing brackets in this order. (b) There are no labels a39a89a36 a26 a27 a25a22 anda77 a36 a26 a42 a25a22 , where a38 a13 a23a79a13 a2 , that are attached to opening brackets in this order.</Paragraph>
    <Paragraph position="11"> (c) There are no labels a39a80a36 a26 a27 a25a22 and a77 a36 a26 a42 a25a22 , wherea38 a13 a23a86a13 a2 , that are attached respectively to a closing bracket and an opening bracket so that the color index of the closing bracket is smaller than that of the opening bracket.</Paragraph>
    <Paragraph position="12"> Axiom 17. There is no label a39 a36 a26a70a27 a25a0 ,a38 a1 a3a92a13 a2 , that is attached to an opening bracket, if on the left of the label a39 there is no a77 a36 a26a28a42 a25a0 , or on the right of the label a39 there is no labela77 a36 a26a70a42 a25a0 a34 a3 .</Paragraph>
    <Paragraph position="13"> Axiom 18. There is no label a39 a36 a26 a27 a25a0 ,a38 a1 a3a92a13 a2 , that is attached to a closing bracket with color a2 , if on the right of the label</Paragraph>
    <Paragraph position="15"> a26 a42 a25a0 , or on the left of the label a39 there is no label a77 a36 a26 a42 a25a0 a34 a3 , or there is no label a77 a36 a26 a42 a25a0 a34 a3 that is attached to an opening bracket with a color greater than a2 .</Paragraph>
    <Paragraph position="16"> Axiom 19. In articulation nodes, the counters of the outgoing arc labels must be zero.</Paragraph>
    <Paragraph position="17"> More formally these are given as follows:  on the top of the bounded nested crossing depth and bounded non-projectivity depth using a carefully designed linear context free rewriting system. A regular approximation for such a grammar is obtained by compiling the colored dependency rules to constraints that specify local subcategorization features (labels and bracket colors) within the node boundaries. The current axioms will take care of the non-local structure of the described D-trees.</Paragraph>
    <Paragraph position="18"> Example 1. The following set of rules describes the dependency tree shown in Figure 1:</Paragraph>
    <Paragraph position="20"> When the second and the third rule, for example, are compiled, we obtain the following two constraints: pred a71 # ]a6 a31venita55a56a16 a33a48 a19 [a7 a37 a36 a7a61a24 subj [a7 a48 adv # subj a71 # gntv ]a7 a48 a19 ]a7 a37 a34 a7a61a24 attr ]a3 a31aetasa55a22a16 a33a48 #a74</Paragraph>
  </Section>
  <Section position="8" start_page="0" end_page="0" type="metho">
    <SectionTitle>
8 Further Work
</SectionTitle>
    <Paragraph position="0"> In the future, the current axiomatization should be extended to allow free dependents, and to include rules without colors and arc order. Furthermore, efficient methods for applying the axioms should be developed and a standard finite-state parser using these axioms should be specified.</Paragraph>
    <Paragraph position="1"> The approach could be extended with a multitiered approach where different kinds of bracketed strings (including e.g. P-markers) are processed with a multi-tape finite automaton. We could also use weighted automata to improve the ranking of alternative analyses.</Paragraph>
    <Paragraph position="2"> We would like to develop full scale grammars and to evaluate the presented representation properly in practical setting. Possibilities to induce a grammar automatically from a treebank could be examined.</Paragraph>
    <Paragraph position="3"> The proposed complexity bounds could be applied also to treebank validation and more generally in linguistic studies of natural language complexity.</Paragraph>
  </Section>
class="xml-element"></Paper>