<?xml version="1.0" standalone="yes"?>
<Paper uid="W98-1203">
  <Title>I Learning a Lexicalized Grammar for German</Title>
  <Section position="3" start_page="0" end_page="0" type="metho">
    <SectionTitle>
2. Link Grammar
</SectionTitle>
    <Paragraph position="0"> Link grammar (Grinberg, Lafferty &amp; Sleator, 1995; Sleator &amp; Temperley 1991) is a highly lexieal, context-free formalism that does not rely on constituent structure. Instead, it models connections between word pairs without building a hierarchical strueture.</Paragraph>
    <Paragraph position="1"> The link grammar formalism is best explained with an example of a linkage (i.e. a link grammar parse): Figure 1 shows a linkage for an English sentence. A linkage is a graph in which the vertices, representing the words, are connected by labeled arcs. These arcs are called links. For a grammatically correct sentence, the linkage must fulfill the following requirements: the links do not cross (= planarity), the graph is connected, and at most one arc connects a pair of words.</Paragraph>
    <Paragraph position="2"> If there is no linkage for a sequence of words, the sentence is not in the language modeled by the grammar.</Paragraph>
    <Paragraph position="4"> Sandra Kiibler (1998) Learning a Lexicalised Grammar for German. In D.M.W. Powers (ed.) NeMLaP3/CoNLL98: New Methods in Language Processing and Computational Natural Language Learning, ACL, pp 11-18.</Paragraph>
    <Paragraph position="5"> The labels on the arcs denote the syntactic relations or constituent relationships of the connected words* In figure 1, the link labeled S connects the subject noun to the finite verb, D connects determiners to their nouns, MV connects the verb to the following prepositional phrase, etc.</Paragraph>
    <Paragraph position="6"> The grammar itself consists of a wordlist in which each word is paired up with all potential linking requirements. Each linking requirement models one usage of the word* A linking requirement, also called a disjunct, is a formal specification of the different connectors, which link with a matching connector of another word, including their direction and order* It is usually represented as a pair of ordered lists: the left list, containing connectors that link to the left of the word, and the right list, containing connectors that link to the fight. For example, the linking requirement of the word &amp;quot;girl&amp;quot; in figure 1 is characterized by the formula ((D, A), (S)), for &amp;quot;finished&amp;quot; the formula is ((S) (O, MV)), and for &amp;quot;young&amp;quot; (0, (A)). In a more sophisticated version of the grammar, the labels are annotated by features, e.g. to ensure agreement between subject and verb.</Paragraph>
    <Paragraph position="7"> The link grammar formalism is similar to ency grammar (Mercuk 1988, Tesrtiere 1959) in that both of them model connections between single words. But link grammar connections are purely lexical: they do not intend to model valency or semantic aspects of words. An additional advantage of link grammar is that there exists an efficient parsing algorithm (Sleator &amp; Temperley 1991, 1996) whereas there does not seem to exist one for dependency grammar.</Paragraph>
    <Paragraph position="8"> 2.1. Adaptations of the Formalism to Cover the German Language Link grammar, like many other formalisms, seems to be especially suited for the English language. When trying to use this formalism for other languages, it seems wise to adapt the formalism to the needs of these languages, most of which are caused by a freer word order. In working with the German language, I have found the following changes immensely helpful: ~.D Mann ~,~MV_ the man laughs often</Paragraph>
  </Section>
  <Section position="4" start_page="0" end_page="0" type="metho">
    <SectionTitle>
MV D
</SectionTitle>
    <Paragraph position="0"> often laughs the man Sleator and Temperley (1991) strongly prefer local links (i.e. links connecting words to their immediate neighbors), even if this is not supported by linguistics. As German uses agreement much more extensively than English, it is necessary to link words according to the agreement requirements rather than because of immediate neighborhood. This approach results in considerably more long distance links.</Paragraph>
    <Paragraph position="1"> In English, the word order is rigidly determined for most parts of the sentence. Sleator and Temperley (1991) use different labels for links that can occur in more than one position (e.g. adverbs) depending on whether they are left or right links. In German, however, due to its freer word order, these phenomena are relatively common. In order to avoid using too many different labels describing the same kind of link but in different order, I have introduced the idea Of control, or rather directionality of links. Each link is marked as either controller (8) or controlled (=). I can thus use the S-link for subjects preceding or following the finite verb, as shown in figure 2.</Paragraph>
    <Paragraph position="2"> The principle of planarity states that links in a linkage must not cross* Sleator and Temperley (1991, I) comment that most sentences of most languages adhere to that principle. Unfortunately, German is one of the languages in which this principle is violated in a number of cases. Some of them are caused by the free word order, some by phenomena like the splitting of the verb: llmen wird vorgeworfen, sie h~itten to them is reproached, they had sieh in Berlin getroffen .</Paragraph>
    <Paragraph position="3"> each other in Berlin met They were reproached for having met in Berlin.</Paragraph>
    <Paragraph position="4"> Ich babe den Mann gesehen, der I have the man seen, who das Buch besitzt the book owns I have seen the man who owns the book*  In the first example, the dative object &amp;quot;ihnen&amp;quot; links to &amp;quot;vorgeworfen&amp;quot; and the f'mite verb &amp;quot;wird&amp;quot; to the period. In the second example, &amp;quot;Mann&amp;quot; links to &amp;quot;der&amp;quot;, the relative pronoun and the finite verb to the past participle &amp;quot;gesehen&amp;quot;.</Paragraph>
    <Paragraph position="5"> As crossing links are inevitable in German, there is a special marker for such links that may cross.</Paragraph>
    <Paragraph position="6"> 2.2. What Advantages Does Link Grammar Offer for Learning? Link grammar offers at least two characteristics that will be of advantage in syntax learning: Instead of relying on a hierarchical constituent structure, the link grammar formalisms is based on links on a single level. Therefore, they can be learned independently; there is no need for a top-down or bottom-up structuring. Thus errors in earlier steps of building the structure cannot have as disastrous effects as with constituent structures.</Paragraph>
    <Paragraph position="7"> Another problem of constituent grammars, which may cause problems in learning, are long-distance dependencies. The information about a gap somewhere in the structure is usually passed on through several levels of the constituent tree. In link grammar, however, these distances are covered by a direct link, which means that these phenomena do not need any special attention during the learning process.</Paragraph>
    <Section position="1" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
2.3. Former Approaches to Learning
Link Grammar
</SectionTitle>
      <Paragraph position="0"> There already exist two approaches to learning with a link grammar formalism (Delia Pietra et al., 1994; Fong &amp; Wu, 1995). In both cases, the probabilistic version of the grammar (Lafferty, Sleator &amp; Temperley, 1992) are used and the word pairs plus their probabilities are inferred from a corpus by an EM-algorithrn. The probabilistie model of link grammar restricts disjuncts in that only one left connector and at most two right connectors are allowed. At least for German, this formalism leads to a very unnatural and counterintuitive description.</Paragraph>
      <Paragraph position="1"> Additionally, to reduce the amount of data to be processed, both approaches did not use the link type information but assumed only one type of link. This restriction may be very helpful concerning computing time yet thus valuable information is not taken into  consideration.</Paragraph>
      <Paragraph position="2"> 3. A Fuzzy Relation for Representing  the Link Grammar Ever since Zadeh (1965) has introduced fuzzy sets, the interest in fuzzy modeling has increased steadily. In computational linguistics, fuzzy methods are mainly used in semantics to model vague meaning like the meaning of the concept &amp;quot;fast&amp;quot;. A fuzzy set representing this concept would give gradually increasing grades of membership to the speed between 0 and 120 mph.</Paragraph>
      <Paragraph position="3"> However, fuzzy methods cannot only be used for modeling vagueness, they are also useful in cases where the given information is either inexact or incomplete. Concerning grammar, and especially learning grammar, the latter case must be assumed. A (complete) link grammar can be represented as a (crisp) relation G among the set W of all words and the set D of all potential disjuncts G: W x L --&gt; {0,1} with its characteristic function gc(w'd)={; /felse &lt;w,d&gt;is grammatical where an ordered pair &lt;w, d&gt; is assigned the membership value 1 if d is a valid linkage for the word w. Now if only a fragment of the grammar is known, the fuzzy relation G* is defined as</Paragraph>
      <Paragraph position="5"> where the membership value does not indicate whether the ordered pair is in the grammar but whether the pair is known to be in the grammar or to what degree it is assumed to be in the grammar (for the characteristic function see section 4.1). Here the value 1 indicates that it is certain that the linkage is valid for the word in question, 0 indicates that there has never been any reason to assume that w takes d as a valid linkage.</Paragraph>
    </Section>
  </Section>
  <Section position="5" start_page="0" end_page="0" type="metho">
    <SectionTitle>
4. Learning the Link Grammar
</SectionTitle>
    <Paragraph position="0"> The system starts with a grammar fragment extractJ~d from a small corpus of 50 annotated sentences. These sentences, as well as the test sentence used below, are taken from the TAZ, a German newspaper. At this stage, the grammar is crisp, i.e. the only membership values used are 1 for pairs of words and disjuncts found in the corpus and 0 otherwise. Then optional elements are marked, i.e. if a word is connected to two disjuncts d and d' of which d is equal to d' except that d has one ore more connector that are not in d', then these connectors are marked as optional.</Paragraph>
    <Paragraph position="1"> The learning process itself is incremental: once a new sentence is presented to the system, the parsing component takes over. It attempts to parse the sentence with the crisp version of the grammar, i.e. with all pairs of words and disjuncts for which the relation G* gives the value I. (At the moment, the parser still has to be implemented. The algorithm is described by Sleator and Temperley (1991,1996) yet it must be modified to account for the changes in the link grammar formalism necessary to describe German.) If the first attempt with the crisp grammar does not succeed, the threshold for G* is lowered from 1 to 0.3 and the attempt is repeated. In this case, less reliable information is used but if the parse succeeds, the validity of the disjuncts used in the parse is corroborated.</Paragraph>
    <Paragraph position="2"> Therefore their membership value is increased.</Paragraph>
    <Paragraph position="3"> If the parser, however, does not succeed in parsing the sentence, the learning component is called: Kfibler 13 Learning Lexicalised Grammar for German * As a first step, every word in the sentence is tagged. (The formalism used for tagging will be Brilrs (1993, 1995) transformation-based error-driven tagger.) Unlike other approaches to learning using constituent-based grammars, this system does not use the wordclass information to restrict the roles, a word can play in the parse. Rather it takes this information as a starting point in the search for potential disjuncts for unknown words. And ifa new disjunct is found for a word already in the grammar, its credibility is tested by comparing the word's wordelass to the wordelass of the word with which the disjunct has the highest membership value in the grammar (el. below). In both cases, the word,lass information is only used to corroborate decisions made in advance.</Paragraph>
    <Paragraph position="4"> * After the wordelass information is provided, the systems looks for every potential conjugated verb in the sentence. For each of these verbs, a partial linkage is constructed, in which the verb is connected to the period by an Xp-link. This is an important step as the Xp-link cannot be crossed by any other link added later in the process.</Paragraph>
    <Paragraph position="5"> * Then for all words listed in the grammar, the system retrieves all disjuncts which are connected to them. With these disjuncts, all potential partial linkages are constructed by linking all words which possess matching connectors. If word x, for example, possesses a disjunct with a connector =Jd-, it will be linked to word y possessing a disjunct with connector SSJd+. All these links must fulfill the conditions that they must not cross, that the order of connectors in the disjunct must not be changed, and that no two links can connect the same pair of words.</Paragraph>
    <Paragraph position="6"> * In the next step, every disjunct in the partial parse which is activated (i.e. partially f'dled) attempts to fill the remaining connectors by linking them to neighboring words without violating the restrictions mentioned above. Like in the previous steps, all potential combinations are stored.</Paragraph>
    <Paragraph position="7"> * After that, all words for which linking information is available but which are not yet connected to the partial parse are linked in any possible way.</Paragraph>
    <Paragraph position="8"> * If the linkage is not connected at this stage, the words left out are either unknown or the disjunct needed for this sentence has not been recorded for them yet. Starting with an initial corpus of only 50 sentences, this will be the case for about 90% of the sentences. But even if the grammar fragment is increased considerably, it will be highly probable that most linkages are not connected at this stage. As the disjuncts needed to complete the linkage, or at least very similar ones, may already be included in the grammar, it is necessary to have an efficient relrieval function.</Paragraph>
    <Paragraph position="9"> In order to reduce the search space, the wordclass information is used to find entries with similar linking requirements. All the disjuncts found in this search are then given to the unknown word as potential disjuncts. They are then used to complete the linkage.</Paragraph>
    <Paragraph position="10"> * At this stage in the process, the learner has aggregated a number of complete linkages. The next task must then be to evaluate them. This is done by the following method: First the membership value for each word and the disjunct used in the linkage is calculated (el. section 4.1). This is not as trivial as it may seem as for many words, the disjuncts actually used in the linkage are different form those originally retrieved from the grammar. If connector could not be filled, they are dropped, while other connectors which originate from the linking requirements of another word are added. From these membership values of the single words, the overall value of the linkage is calculated as the arithmetic mean. This final figure is used as a measure of the quality of the linkage.</Paragraph>
    <Paragraph position="11"> * The best parse then is given as the preferred parse for the input sentence, and all new pairs of words and disjuncts are ad~d to the grammar with their calculated membership values. For pairs already in the fuzzy grammar, the membership value is increased.</Paragraph>
    <Paragraph position="12"> * As a last step, for every new or modified word, optional elements are marked in the disjuncts.</Paragraph>
    <Section position="1" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
4.1. Calculating the Membership Value
</SectionTitle>
      <Paragraph position="0"> The following algorithm is used to calculate the membership value lx(w,d) for the pair &lt;w, d&gt;.</Paragraph>
      <Paragraph position="2"> else get the pair &lt;w',d'&gt; with wordclass(w) wordclass(w') and minimal distance(d, d') and maximal ~.a.(W ,d' )then</Paragraph>
      <Paragraph position="4"> then get the pair &lt;w',d'&gt; with (wordclass(w) m_ wordelass(w')) and minimal distance(d, d') and maximal p c. (w', d' ), then</Paragraph>
      <Paragraph position="6"> (0~ (SSJd, =MVp)), (0, (SSJd, =Yz, =MVp)), ((=MVp), (SSJdp)), ((=Mp), (SSJd)), ((MVpv), (SSJd)) (0., (=Dsfdn)), ((=Ons), (SSGEp+)) ((SSMVp), (SSSpl, SSIn, SSXk, =Coq)), ((SSSial,), (SSIn, SSXk, SSCOq)), ((SSRSrp3), (~In))</Paragraph>
      <Paragraph position="8"> Exception: Nothing is a,~d if the connector c is the same as the preceding connector and the connector can be found in G* at least once marked for multiple occllrrence.</Paragraph>
      <Paragraph position="9"> The reason why the disjunct is punished harder for missing controlled links is that optional connectors usually are controlling.</Paragraph>
      <Paragraph position="10"> 4.2. Example In this section, we will look at an example sentence. It will not be possible to give all the potential linkages but the gist of the argument should become clear.</Paragraph>
      <Paragraph position="11"> The example sentence is: Aber yon einer Fehlemfihnmg kOnnenwir but of a malnutrition can we heute schon sprechen today ak-eadyspeak Table 1 gives the information that can be extracted from the initial grammar G*. All the disjuncts listed for a word have the membership value 1 concerning this word. As can be seen in the table, there is only one unknown word in the sentence. However, only for the words &amp;quot;yon&amp;quot;, &amp;quot;einer&amp;quot;, &amp;quot;wir&amp;quot;, &amp;quot;heute&amp;quot;, and .... schon, the needed disjunct is listed. All words belonging to an open wordclass except &amp;quot;wir&amp;quot; give only partial or no information needed for this sentence.</Paragraph>
      <Paragraph position="12">  1. step: The only wordclass information needed in the further process is that &amp;quot;Fehlemfihrung&amp;quot; is a noun, and &amp;quot;k6nnen&amp;quot; and &amp;quot;sprechen&amp;quot; are potential verbs. 2. step: As we know from step 1, both &amp;quot;kOnnen&amp;quot; and &amp;quot;sprechen&amp;quot; are verbs. So there are two ways to (c ~d)^(C Ed') place the first link, linking each verb in turn to the , period by an Xp-link.</Paragraph>
      <Paragraph position="13"> 0nly features(d) ~ features(d / 3. step: For the information given in G*, see table contr01(d)-- SS' 1. Three potential linkages are shown in figure 3. For each given linkage, there is another one differing only control(d)--' =' by linking &amp;quot;schon&amp;quot; instead of&amp;quot;beute&amp;quot; to &amp;quot;sprechen&amp;quot;. 4. step: There are too many possibilities to link the  remaining connectors of activated disjuncts to their neighbor. Figure 4 shows three of them, randomly chosen.</Paragraph>
      <Paragraph position="14"> 5. step: In figure 5, only two potential linkages are given at~er the remaining words are connected, the overall membership value for these linkages is calculated in step 7.</Paragraph>
      <Paragraph position="15">  6. step: This step is not necessary because the linkage is complete.</Paragraph>
      <Paragraph position="16"> 7. step: The calculations for the linkages represented in figure 5 are given in table 2 and 3 respectively. null 8. step: The disjuncts from table 2 for the words &amp;quot;aber&amp;quot;, &amp;quot;Fehlem/ihrung&amp;quot;, &amp;quot;k6nnen&amp;quot;, and &amp;quot;sprechen&amp;quot; with their membership values are Mded~ to the grarrmlar.</Paragraph>
      <Paragraph position="17"> 9. step: There are two new disjuncts which can be  marked for optional connectors: For the word &amp;quot;aber&amp;quot;, the new disjunct is (({=CC}, {=Xk}), (SSCd)), and for &amp;quot;kOnnen&amp;quot; (({=Cd}, SSMVp), (SSSpl, SSIn, {Xp})).</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>