<?xml version="1.0" standalone="yes"?>
<Paper uid="A00-2014">
  <Title>The Effectiveness of Corpus-Induced Dependency Grammars for Post-processing Speech*</Title>
  <Section position="4" start_page="102" end_page="104" type="metho">
    <SectionTitle>
3 Learning CDG Rules
</SectionTitle>
    <Paragraph position="0"> In this section, we introduce CDG and then describe how CDG constraints can be learned from sentences annotated with grammatical information.</Paragraph>
    <Section position="1" start_page="102" end_page="103" type="sub_section">
      <SectionTitle>
3.1 Introduction to CDG
</SectionTitle>
      <Paragraph position="0"> Constraint Dependency Grammar (CDG), first introduced by Maruyama (Maruyama, 1990a; Maruyama, 1990b), uses constraints to determine the grammatical dependencies for a sentence. The parsing algorithm is framed as a constraint satisfaction problem: the rules are the constraints and the solutions are the parses. A CDG is defined as a five-tuple, (2E, R, L, C, T), where ~ = {al,..., c%} is a finite set of lexical categories (e.g., determiner), R = {rl,...,rp} is a finite set of uniquely named roles or role ids (e.g., governor, needl, need2), L = {ll,...,lq} is a finite set of labels (e.g., subject), C is a constraint formula, and T is a table that specifies allowable category-role-label combinations.</Paragraph>
      <Paragraph position="1"> A sentence s - WlW2W3...wn has length n and is an element of ~*. For each word wi E ~ of a sentence s, there are up to p different roles (with most words needing only one or two (Harper et al., 1999a)), yielding a maximum of n * p roles for the entire sentence. A role is a variable that is assigned a role value, an element of the set L x (1, 2,..., n}.</Paragraph>
      <Paragraph position="2"> Role values are denoted as l-m, where l E L and m E (1, 2,..., n} is called the modifiee. Maruyama originally used a modifiee of NIL to indicate that a role value does not require a modifiee, but it is more parsimonious to indicate that there is no dependent by setting the modifiee to the position of its word.</Paragraph>
      <Paragraph position="3"> Role values are assigned to roles to record the syntactic dependencies between words in the sentence.</Paragraph>
      <Paragraph position="4">  The governor role is assigned role values such that the modifiee of the word indicates the position of the word's governor or head (e.g., DET-3, when assigned to the governor role of a determiner, indicates its function and the position of its head). Every word in a sentence has a governor role. Need roles are used to ensure the requirements of a word are met.</Paragraph>
      <Paragraph position="5"> For example, an object is required by a verb that subcategorizes for one, unless it has passive voice.</Paragraph>
      <Paragraph position="6"> The required object is accounted for by requiring the verb's need role to be assigned a role value with a modifiee that points at the object. Words can have more than one need role, depending on the lexical category of the word. The table T indicates the roles that a word with a particular lexical category must support.</Paragraph>
      <Paragraph position="7"> A sentence s is said to be generated by the grammar G if there exists an assignment A that maps a role value to each of the roles for s such that C is satisfied. There may be more than one assignment of role values to the roles of a sentence that satisfies C, in which case there is ambiguity. C is a first-order predicate calculus formula over all roles that requires that an assignment of role values to roles be consistent with the formula; those role values inconsistent with C can be eliminated. A subformula P~ of C is a predicate involving =, &lt;, or &gt;, or predicates joined by the logical connectives and, or, if, or not. A subformula is a unary constraint if it contains only a single variable (by convention, we use zl) and a binary constraint if it contains two variables (by convention zl and z2). An example of a unary and binary constraint appears in Figure 2.</Paragraph>
      <Paragraph position="8"> A CDG has an arity parameter a, which indicates the maximum number of variables in the subformulas of C, and a degree parameter d, which is the number of roles in the grammar. An arity of two suffices to represent a grammar at least as powerful as a context-free grammar (Maruyama, 1990a; Maruyama, 1990b). In (Harper et al., 1999a), we developed a way to write constraints concerning the category and feature values of a modifiee of a role value (or role value pair). These constraints loosely capture binary constraint information in unary constraints (or beyond binary for binary constraints) and results in more efficient parsing.</Paragraph>
      <Paragraph position="9"> A u, liwy C/~nst\]llnt requiring that * role vmluo IlmlgnlKI to the vernot role of I determiner have the label D~ lind * modlflee pointing to * lub4NIqtNl~t wocd* (if (and (= (category x 1) determiner)</Paragraph>
      <Paragraph position="11"> A binary oonatrllnt requiring that * role vllue with the libel S Illlgned to * ne~dl role of one word pOklt It Imother word whole governor role I= mml~gnened * role veltm with the libel 8UBJ and * rnodlflee that point* beck at the flrat word.</Paragraph>
      <Paragraph position="12"> (if (and (= (label x I ) S)</Paragraph>
      <Paragraph position="14"> The white box in Figure 3 depicts a parse for the sentence Clear the screen from the Resource Management corpus (Price et al., 1988) (the ARV and ARVP in the gray box will be discussed later), which is a corpus we will use to evaluate our speech processing system. We have constructed a conventional CDG with around 1,500 unary and binary constraints (i.e., its arity is 2) that were designed to parse the sentences in the corpus. This CDG covers a wide variety of grammar constructs (including conjunctions and wh-movement) and has a fairly rich semantics. It uses 16 lexical categories, 4 roles (so its degree is 4), 24 labels, and 13 lexical feature types (subcat, agr, case, vtype (e.g., progressive), mood, gap, inverted, voice, behavior (e.g., mass), type (e.g., interrogative, relative), semtype, takesdet, and conjtype). The parse in Figure 3 is an assignment of role values to roles that is consistent with the unary and binary constraints. A role value, when assigned to a role, has access to not only the label and modifiee of its role value, but also the role name of the role to which it is assigned, information specific to the word (i.e., the word's position in the sentence, its lexical category, and feature values for each feature), and information about the lexical class and feature values of its modifiee. Our unary and binary constraints use this information to eliminate ungrammatical assignments.</Paragraph>
      <Paragraph position="15"> Parse for &amp;quot;Clear the screen&amp;quot;</Paragraph>
      <Paragraph position="17"> sented by the assignment of role values to roles associated with a word with a specific lexical category and one feature value per feature. ARVs and ARVPs (see gray box) represent grammatical relations that can be extracted from a sentence's parse.</Paragraph>
    </Section>
    <Section position="2" start_page="103" end_page="104" type="sub_section">
      <SectionTitle>
3.2 Learning CDG Constraints
</SectionTitle>
      <Paragraph position="0"> The grammaticality of a sentence in a language defined by a CDG was originally determined by applying the constraints of the grammar to the possible  role value assignments. If the set of all possible role values assigned to the roles of a sentence of length n is denotedS1 =Y;.x RxPOSxLxMODx Ftx ... x Fk, where k is the number of feature types, Fi represents the set of feature values for that type, POS = {1, 2,..., n} is the set of possible positions, MOD = {1, 2,..., n} is the set of possible modiflees, and n is sentence length (which can be any arbitrary natural number), then unary constraints partition $1 into grammatical and ungrammatical role values. Similarly, binary constraints partition the set $2 = $1 x $1 = S~ into compatible and incompatible pairs. Building upon this concept of role value partitioning, it is possible to construct another way of representing unary and binary constraints because CDG constraints do not need to reference the exact position of a word or a modifiee in the sentence to parse sentences (Harper and Helzerman, 1995; Maruyama, 1990a; Maruyama, 1990b; Menzel, 1994; Menzel, 1995).</Paragraph>
      <Paragraph position="1"> To represent the relative, rather than the absolute, position information for the role values in a grammatical sentence, it is only necessary to represent the positional relations between the modifiees and the positions of the role values. To support an arity of 2, these relations involve either equality or less-than relations over the modifiees and positions of role values assigned to the roles zl and x2. Since unary constraints operate over role values assigned to a single role, the only relative position relations that can be tested are between the role value's position (denoted as Pzl) and its modifiee (denoted as Mzl); one and only one of the following three relations must be true: (P~:I &lt; Mzl), (Mzl &lt; Pzl), or (Pzl = Mzl). Since binary constraints operate over role values assigned to pairs of roles, zl and z2, the only possible relative position relations that can be tested are between Pzl and Mxt, P:e2 and Mx2, Pzl and Mz~, Pz2 and Mxt, Pzt and Px2, Mxl and Mz2. Note that each of the six has three positional relations (as in the case of unary relations on Pzl and Mzt) such that one and only one of them is simultaneously true.</Paragraph>
      <Paragraph position="2"> The unary and binary positional relations provide the necessary mechanism to develop an alternative view of the unary and binary constraints. First, we develop the concept of an abstract role value (ARV), which is a finite characterization of all possible role values using relative, rather than absolute, position relations. Formally, an ARV for a particular grammar G = (~,, R, L, C, T, Ft,..., Fk) is an element of the set: .dl = ExRx L xFt x...xFkxUC, where UC encodes the three possible positional relations between Pxl and Mxl. The gray box of Figure 3 shows an example of an ARV obtained from the parsed sentence. Note that .At is a finite set representing the space of all possible ARVs for the grammar1; hence, the set provides an alternative characterization of the unary constraints for the grammar, which can be partitioned into positive (grammatical) and negative (ungrammatical) ARVs. During parsing, if a role value does not match one of the elements in the positive ARV space, then it should be disallowed.</Paragraph>
      <Paragraph position="3"> Positive ARVs can be obtained directly from the parses of sentences: for each role value in a parse for a sentence, simply extract its category, feature, role, and label information, and then determine the positional relation that holds between the role value's position and modifiee.</Paragraph>
      <Paragraph position="4"> Similarly the set of legal abstract role value pairs</Paragraph>
      <Paragraph position="6"> relations among Pxl, Mxt, Px2, and Mx2, provides an alternative definition for the binary constraints 2.</Paragraph>
      <Paragraph position="7"> The gray box of Figure 3 shows an example of an ARVP obtained from the parsed sentence. Positive ARVPs can be obtained directly from the parses of sentences. For each pair of role values assigned to different roles, simply extract their category, feature, role, and label information, and then determine the positional relations that hold between the positions and modifiees.</Paragraph>
      <Paragraph position="8"> An enumeration of the positive ARV/ARVPs can be used to represent the CDG constraints, C, and ARV/ARVPs are PAC-learnable from positive examples, as can be shown using the techniques of (Natarajan, 1989; Valiant, 1984). ARV/ARVP constraints can be enforced by using a fast table lookup method to see if a role value (or role value pair) is allowed (rather than propagating thousands of constraints), thus speeding up the parser.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>