<?xml version="1.0" standalone="yes"?>
<Paper uid="W06-3508">
  <Title>Searching for Grammar Right</Title>
  <Section position="3" start_page="57" end_page="57" type="metho">
    <SectionTitle>
2 Grammar Formalism
</SectionTitle>
    <Paragraph position="0"> The most crucial foundation that is needed to build a grammar learning system is a grammar formalism. Therefore, we are designing a new formalization of construction grammar called ECtoloG (Porzel et al., 2006; Micelli et al., in press).</Paragraph>
    <Paragraph position="1"> One existing formal computational model of construction grammar is the Embodied Construction Grammar (ECG) (Chang et al., 2002; Bergen and Chang, 2002), with its main focus being on language understanding and later simulation</Paragraph>
  </Section>
  <Section position="4" start_page="57" end_page="57" type="metho">
    <SectionTitle>
. A
</SectionTitle>
    <Paragraph position="0"> congruent and parallel development has led to FCG which simulates the emergence of language (Steels, 2005). FCG is mainly based on the same primitives and operators as ECG is. We decided to employ ECG in our model mainly for historical reasons (see details about its development in the following section), adhering to its main primitives and operators, but employing the state of the art in knowledge representation. We adopt insights and mechanisms of FCG where applicable.</Paragraph>
    <Section position="1" start_page="57" end_page="57" type="sub_section">
      <SectionTitle>
2.1 Construction Grammar and ECG
</SectionTitle>
      <Paragraph position="0"> One main difference between West Coast Grammar (Langacker, 1987; Lakoff, 1987) and East Coast Grammar (Chomsky, 1965; Katz, 1972) is the fact that construction grammar offers a vertical - not a horizontal - organisation of any knowledge concerning a language's grammar. That is, that generative grammars split form from function.</Paragraph>
      <Paragraph position="1"> Syntax, morphology, a lexicon or other formal components of the grammar constitute form, while the conventional function is defined by semantics.</Paragraph>
      <Paragraph position="2"> All constructions of a language, however, form in Langacker's terms &amp;quot;a structured inventory of conventional linguistic units&amp;quot; (Langacker, 1987:54). This inventory is network-structured, i.e.</Paragraph>
      <Paragraph position="3"> there are at least taxonomic links among the constructions (Diessel, 2004). This structure presents  For a detailed ECG analysis of a declarative utterance, i.e. the sentence Harry walked into the cafe, see Bergen and Chang (2002).</Paragraph>
      <Paragraph position="4"> one of the main differences between generative and construction grammars (Croft, to appear). One of the most cited examples that evidences the necessity, that there can be no explicit separation between syntax and semantics, is Goldberg's example sentence (Goldberg, 1995:29): (1) he sneezed the napkin off the table.</Paragraph>
      <Paragraph position="5"> The whole meaning of this sentence cannot be gathered from the meanings of the discrete words.</Paragraph>
      <Paragraph position="6"> The direct object the napkin is not postulated by the verb to sneeze. This intransitive verb would have three arguments in a lexico-semantic theory: 'X causes Y to move Z by sneezing'. Goldberg states that the additional meaning of caused motion which is added to the conventional meaning of the verb sneeze is offered by the respective caused-motion construction. Based on this background ECG - a formal computational model of construction grammar - was developed within the Neural Theory of Language project (NTL) and the EDU project (EDU).</Paragraph>
      <Paragraph position="7"> While other approaches consider language as completely independent from the organism which uses it, ECG claims that several characteristics of the user's sensorimotor system can influence his or her language (Gallese and Lakoff, 2005). The needed dynamic and inferential semantics in ECG is represented by embodied schemas. These schemas are known under the term of image schemas in traditional cognitive semantics and constitute schematic recurring patterns of sensorimotor experience (Johnson, 1987; Lakoff, 1987).</Paragraph>
      <Paragraph position="8"> The current ASCII format of ECG is insufficient for building scalable NLU systems in the long run. Therefore, our attempt at formalizing construction grammar results in an ontological model that combines two ontological modeling frameworks endowed with a construction grammar layer, based on the main ideas behind ECG. The following section describes the resulting ontology, pointing out main challenges and advantages of that approach.</Paragraph>
    </Section>
  </Section>
  <Section position="5" start_page="57" end_page="61" type="metho">
    <SectionTitle>
3 Formalizing Construction Grammar
</SectionTitle>
    <Paragraph position="0"> The ontological frameworks mentioned above are Descriptions &amp; Situations (D&amp;S) (Gangemi and Mika, 2003) and Ontology of Information Objects (OIO) (Guarino, 2006), which both are extensions of the Descriptive Ontology for Linguistic and  Cognitive Engineering (DOLCE) (Masolo et al., 2003).</Paragraph>
    <Paragraph position="1"> D&amp;S is an ontology for representing a variety of reified contexts and states of affairs. In contrast to physical objects or events, the extensions of ontologies to the domain of non-physical objects pose a challenge to the ontology engineer. The reason for this lies in the fact that non-physical objects are taken to have meaning only in combination with some other ground entity. Accordingly, their logical representation is generally set at the level of theories or models and not at the level of concepts or relations (see Gangemi and Mika, 2003). It is, therefore, important to keep in mind that the meaning of a given linguistic expression emerges only through the combination of both linguistic and conceptual knowledge with &amp;quot;basic&amp;quot; ontological knowledge, as modeled in such ground ontologies.</Paragraph>
    <Paragraph position="2"> Next to the support via dedicated editors and inference engines, one of the central advantages of our ensuing ontological model over the currently used ASCII-format of ECG lies in its compatibility with other ground ontologies developed within the Semantic Web framework.</Paragraph>
    <Section position="1" start_page="58" end_page="58" type="sub_section">
      <SectionTitle>
3.1 Modeling of Constructions
</SectionTitle>
      <Paragraph position="0"> Constructions are modeled in the ECtoloG as information-objects. According to the specification of the OIO, information objects have - amongst others - the following properties: They are social objects realizable by some entity and they can express a description, which represents in this ontology the ontological equivalent of a meaning or a conceptualization. Since a construction constitutes a pairing of form and meaning according to the original theory of construction grammar, both properties are of advantage for our ontological model. To keep the construction's original structure, the form pole can be modeled with the help of the realized-by property  while the meaning pole is built via the edns:expresses property. Both processes are described more detailed in the following section.</Paragraph>
    </Section>
    <Section position="2" start_page="58" end_page="58" type="sub_section">
      <SectionTitle>
Holophrastic Constructions
</SectionTitle>
      <Paragraph position="0"> The class of lexical constructions is modeled as a subclass of referringConstruction. Since it is a  For more details see Porzel et al. (2006).  We adhere to the convention to present both ontological properties, classes, and instances in italics. subclass of the class information-object it inherits the edns:expresses property. The referringConstruction class has a restriction on this property that denotes, that at least one of the values of the edns:expresses property is of type schema. Modeling this restriction is done by means of the built-in owl:someValuesFrom constraint. The restriction counts for all constructions that express a schema. It has no effect on the whole class of constructions, i.e. it is possible that there exist constructions that do not express a single schema, as e.g. compositional ones, whose meaning is a composite of all constructions and schemas that constitute that compositional construction.</Paragraph>
      <Paragraph position="1"> The form pole of each construction is modeled with the help of the realized-by property. This property designates that a (physical) representation - as e.g. the orthographic form of the construction - realizes a non-physical object - in this case our construction. This property is also inherited from the class information-object, the superclass of constructions. What fills the range of that property is the class of edns:physical-realization. Therefore, we define an instance of inf:writing, which then fills the form pole of the respective construction.</Paragraph>
      <Paragraph position="2"> This instance has once more a relation which connects it to instances of the class inf:word which designate the realization of the instance of the inf:writing class.</Paragraph>
      <Paragraph position="3"> This way of modeling the form pole of each lexical construction enables us to automatically populate our model with new instances of constructions, as will be described more detailed in section 5.1.</Paragraph>
      <Paragraph position="4"> Analogous to the modeling of meaning in the original ECG, the meaning pole is 'filled' with an instance of the class of image schema. This can be done with the help of the edns:expresses relation.</Paragraph>
      <Paragraph position="5"> This relation is defined, according to the specification of the D&amp;S ontology, as a relation between information objects that are used as representations (signs) and their content, i.e. their meaning or conceptualization. In this ontology, content is reified as a description, which offered us the possibility to model image schemas as such. How image schemas are modeled will be described in section 3.2.</Paragraph>
    </Section>
    <Section position="3" start_page="58" end_page="59" type="sub_section">
      <SectionTitle>
Compositional Constructions
</SectionTitle>
      <Paragraph position="0"> Compositional constructions are constructions which are on a higher level of abstraction than holophrastic ones. This means, that there exist constructions which combine different constructions  into one unit. ECG designed a so-called constructional block, wherein several constructions are subsumed under and accessible in one more complex construction.</Paragraph>
      <Paragraph position="1"> An example is the DetNoun construction, which combines a determiner and a noun to form one unit. There is the possibility to model different constraints both in the form pole and in the meaning pole of a construction. A form constraint applying to this exact construction is determining that the determiner comes before the noun. This understanding of before corresponds to Allen's definition of his interval relations (Allen, 1983), which states that they don't necessarily have to follow each other but that there could be some modifiers in between the two components of this construction. null A meaning constraint of this construction determines, that the meaning of the noun, used in this respective construction, is assigned to the meaning of the resulting complex construction.</Paragraph>
      <Paragraph position="2">  To be able to represent these phenomena, we firstly defined a class construction-parameter, that denotes a sub-class of edns:parameter, a subclass of edns:concept. There is a property restriction on the class that states that all values of the requisite-for property have to be of type construction. This determines instances of the class construction-parameter to be used only in constructions on a higher level of abstraction. All constructions used on level 0 of a grammar  , i.e. lexical constructions, are at the same time instances of the class construction-parameter so that they can be used in more abstract constructions. The form and meaning constraints still need to be modeled in our framework. To determine which constructions are used in which more abstract construction, new properties are defined. These properties are subproperties of the requisite-for property. An example is the requisite-detnoun-akk-sg property. This property defines that the accusative singular determiner construction and the corresponding noun construction can be requisite-for the compositional construction that combines these two lexical constructions into one noun phrase.</Paragraph>
      <Paragraph position="3">  For further information about which operators are used to model these features in ECG we refer to Bergen and Chang (2002), Chang et al. (2002) and Bryant (2004).</Paragraph>
      <Paragraph position="4">  Following Bryant's (2004) division of constructions into 5 levels of different degrees of schematicity.</Paragraph>
    </Section>
    <Section position="4" start_page="59" end_page="59" type="sub_section">
      <SectionTitle>
3.2 Modeling of Image Schemas
</SectionTitle>
      <Paragraph position="0"> Following Johnson and Lakoff (Johnson, 1987; Lakoff and Johnson, 1980; Lakoff, 1987) image schemas are schematic representations that capture recurrent patterns of sensorimotor experience. According to ECG, a schema is a description whose purpose is filling the meaning pole of a construction. It consists of a list of schematic roles that can serve as simulation parameters.</Paragraph>
      <Paragraph position="1"> In ECG, schemas can be evoked by or can evoke other schemas, i.e. particular schematic-roles of another schema can be imported. A schema can, therefore, be defined against the background of another schema  . The property evokes and its inverse property evoked-by have been defined as subproperties of the dol:generically-dependent-on property and its inverse property dol:genericdependent respectively. Generic dependence is defined in the DOLCE ontology as the dependence on an individual of a given type at some time.</Paragraph>
      <Paragraph position="2"> The class of image schemas is modeled as a sub-class of edns:description (see definition of description in 3.1), in order to enable being employed in the meaning pole of constructions.</Paragraph>
    </Section>
    <Section position="5" start_page="59" end_page="60" type="sub_section">
      <SectionTitle>
Schematic Roles
</SectionTitle>
      <Paragraph position="0"> The class of schematic-roles is a subclass of the edns:concept class. In the specification of D&amp;S a concept is classified as a non-physical object which again is defined by a description. Its function is classifying entities from a ground ontology in order to build situations that can satisfy the description. Schematic roles are parameters that allow other schemas or constructions to refer to the schema's key variable features, e.g. the role of a trajector in a Trajector Landmark-Schema can be played by the same entity that denotes the mover in e.g. a caused-motion schema.</Paragraph>
      <Paragraph position="1"> At the moment, they are modeled with the help of the edns:defines property. A schema defines its schematic roles with this property, denoting a subproperty of the edns:component property. According to the D&amp;S specification, a component is a proper part with a role or function in a system or a context. It is also stated, that roles can be different for the same entity, and the evaluation of them changes according to the kind of entity. This means, that instances of the class schema and its  To clarify this claim see Langacker's hypotenuse example (Langacker, 1987:183ff.).</Paragraph>
      <Paragraph position="2">  subclasses can have instances of the class schematic-role as their components. The schematic-roles class has to fulfil the necessary condition, that at least one of the values of the edns:definedby property is of type schema.</Paragraph>
      <Paragraph position="3"> The domain of the defines property is a description (which can be our schemas) and its range is set to either concepts or figures (which are our schematic roles). The problem occurring hereby is that the roles cannot be filled by complete classes which is necessary in a lot of cases, since the parameters are not always filled with atomic values but possibly with whole classes of entities. Therefore, one could think about modeling schematic roles as properties, setting the domain on the corresponding schema class and the range on the corresponding class whose subclasses and instances can possibly fill its range.</Paragraph>
    </Section>
    <Section position="6" start_page="60" end_page="60" type="sub_section">
      <SectionTitle>
3.3 Linguistic Information
</SectionTitle>
      <Paragraph position="0"> Since linguistic information as e.g. grammatical gender, its case, or the part-of-speech of a word is needed for analyzing natural language texts, this information has to be modeled, as well, in the ECtoloG. Therefore, we integrated the LingInfo model (Buitelaar et al., 2006) into the ECtoloG.</Paragraph>
      <Paragraph position="1"> LingInfo constitutes an ontological model that provides other ontologies with linguistic information for different languages, momentarily for English, French, and German. Main objective of this ontology is to provide a mapping between ontological concepts and lexical items. That is, that the possibility is offered to assign linguistic information as e.g. the orthographic term, its grammatical gender, its part-of-speech, stem etc. to classes and properties. For our purposes, the LingInfo ontology had to be converted from RDFS into OWL-DL format and then integrated into the ECtoloG. For that reason, a new subclass of owl:class was defined: ClassWithLingInfo. Instances of this metaclass are linked through the linginfo property to LingInfo classes. The LingInfo class is used to associate a term, a language, and morphosyntactic information to classes from the ground ontology; e.g. a class CafeConstruction, which is an instance of ClassWithLingInfo, from an ontology proper, can be associated through the property linginfo with Cafe, an instance of the class LingInfo. Thus, the information that the term is German, its part-of-speech is noun and its grammatical gender neuter is obtained.</Paragraph>
      <Paragraph position="2"> Following this approach, our classes of lexical constructions were defined as subclasses of ClassWithLingInfo, being thereby provided with all the necessary linguistic information as defined above. The central challenge resulting from this approach is, that through the definition of a metaclass the ontological format is no longer OWL-DL but goes to OWL-Full which thwarts the employment of Description Logic reasoners. Reasoning will not stay computable and decidable. Future work will address this challenge by means of intertwining the LingInfo model with the ECtoloG grammar model in such a way, that the computational and inferential properties of OWL-DL remain unchallenged.</Paragraph>
      <Paragraph position="3"> Another possibility could be obtaining linguistic information for lexical items through an external lexicon.</Paragraph>
      <Paragraph position="4"> 4 The Web as a Corpus The Seed Corpus C: The primary corpus C in this work is the portion of the World Wide Web confined to web pages containing natural language texts on soccer. To extract natural language texts out of web documents automatically we are using wrapper agents that fulfil this job (see Porzel et al., 2006). Our first goal is to build a grammar that can deal with all occurring language phenomena - i.e.</Paragraph>
      <Paragraph position="5"> both holophrastic and compositional ones - contained in that corpus C.</Paragraph>
      <Paragraph position="6"> Corpus C': Next step is the development of a corpus C', where C' = C + e and e is constituted by a set of new documents. This new corpus is not designed in an arbitrary manner. We search similar pages, adding add them to our original corpus C, as we expect the likelihood of still pretty good coverage together with some new constructions to be maximal, thereby enabling our incremental learning approach. The question emerging hereby is: what constitutes a similar web page? What, therefore, has to be explored are various similarity metrics, defining similarity in a concrete way and evaluate the results against human annotations (see Papineni et al., 2002).</Paragraph>
    </Section>
    <Section position="7" start_page="60" end_page="61" type="sub_section">
      <SectionTitle>
4.1 Similarity Metric
</SectionTitle>
      <Paragraph position="0"> To be able to answer the question which texts are actually similar, similarity needs to be defined precisely. Different approaches could be employed,  i.e. regarding similarity in terms of syntactic or semantic phenomena or a combination of both.</Paragraph>
      <Paragraph position="1"> Since construction grammar makes no separation between syntax and semantics, phenomena that should be counted are both constructions and image schemas. As for holophrastic constructions this presents less of a challenge, we rather expect counting compositional ones being a 'tough cookie'.</Paragraph>
      <Paragraph position="2"> To detect image schemas in natural text automatically, we seek to employ different methodologies, e.g. LSA (Kintsch and van Dijk, 1978), using synonym sets (Fellbaum, 1998) or other ontologies, which could assist in discovering the semantics of an unknown word with its corresponding schematic roles and the appropriate fillers. This or a similar methodology will be applied in the automatic acquisition process as well.</Paragraph>
      <Paragraph position="3"> Another important point is that some terms, or some constructions, need to get a higher relevance factor than others, which will highly depend on context. Such a relevance factor can rank terms or constructions according to their importance in the respective text. Ranking functions that can be examined are, e.g., the TF/IDF function (e.g. Salton, 1989) or other so called bag of words approaches.</Paragraph>
      <Paragraph position="4"> Term statistics in general is often used to determine a scalable measure of similarity between documents so it is said to be a good measure for topical closeness. Also part-of-speech statistics could be partly helpful in defining similarity of documents based on the ensuing type/token ratio.</Paragraph>
      <Paragraph position="5"> The following five steps need to be executed in determining the similarity of two documents: Step 1: Processing of the document D; analyzing the text and creating a list of all occurring words, constructions and/or image schemas. We assume that the best choice is counting constructions and corresponding image schemas, since they represent the semantics of the given text.</Paragraph>
      <Paragraph position="6">  synonyms of sources as e.g. WordNet (Fellbaum, 1998).</Paragraph>
      <Paragraph position="7"> Step 5: Calculating the documents' similarity; defining a threshold up to which documents are considered as being similar. If a document is said to be similar, it is added to the corpus, which becomes the new corpus C'.</Paragraph>
      <Paragraph position="8"> Analysis of the New Corpus C': The new corpus C' is analyzed, whereby the coverage results in coverage A of C' where: A = 100% - (dh + dc) dh denotes all the holophrastic phenomena and dc all compositional phenomena not observed in C.</Paragraph>
    </Section>
  </Section>
  <Section position="6" start_page="61" end_page="62" type="metho">
    <SectionTitle>
5 Grammar Learning
</SectionTitle>
    <Paragraph position="0"> To generate a grammar that covers this new corpus C' different strategies have to be applied for holophrastic items dh which are lexical constructions in our approach and for compositional ones dc meaning constructions on a higher level of abstraction as e.g. constructions that capture grammatical phenomena such as noun phrases or even whole sentences.</Paragraph>
    <Section position="1" start_page="61" end_page="62" type="sub_section">
      <SectionTitle>
5.1 Learning Lexical Constructions
</SectionTitle>
      <Paragraph position="0"> Analogous to the fast mapping process (Carey, 1978) of learning new words based on exposure without additional training or feedback on the correctness of its meaning, we are employing a method of filling our ontology with whole paradigms of new terms  , enabled through the modeling of constructions described in 3.1. First step herein is employing a tool - Morphy (Lezius, 2002) - that enables morphological analysis and synthesis. The analysis of a term yields information about its stem, its part-of-speech, its case, its number, and its grammatical gender. This information can then easily be integrated automatically into the ECtoloG.</Paragraph>
      <Paragraph position="1"> As already mentioned in section 4.3, we are not only trying to automatically acquire the form pole of the constructions, but also its image schematic meaning, that means the network of the schemas that hierarchically form the meaning pole of such a term, applying ontology learning mechanisms (e.g. Loos, 2006) and methods similar to those described in section 4.3. Additionally, investigations are underway to connect the grammar learning framework proposed herein to a computer vision system that provides supplementary feedback con- null We are aware of the fact that fast mapping in humans is limited to color terms, shapes or texture terms, but are employing the method on other kinds of terms, nevertheless, since the grammar learning paradigm in our approach is still in its baby shoes.</Paragraph>
      <Paragraph position="2">  cerning the hypothesized semantics of individual forms in the case of multi-media information.</Paragraph>
    </Section>
    <Section position="2" start_page="62" end_page="62" type="sub_section">
      <SectionTitle>
5.2 Learning Compositional Constructions
</SectionTitle>
      <Paragraph position="0"> Learning of compositional constructions still presents an issue which has not been accounted for, yet. What has already been proposed (Narayanan, inter alia) is that we have to assume a strong inductive bias and different learning algorithms, as e.g. some form of Bayesian learning or model merging (Stolcke, 1994) or reinforcement learning (Sutton and Barto, 1998).</Paragraph>
      <Paragraph position="1"> Another important step that has to be employed is the (re)organization of the so-called constructicon, i.e. our inventory of constructions and schemas.</Paragraph>
      <Paragraph position="2"> These need to be merged, split or maybe thrown out again, depending on their utility, similarity etc.</Paragraph>
    </Section>
    <Section position="3" start_page="62" end_page="62" type="sub_section">
      <SectionTitle>
5.3 Ambiguity
</SectionTitle>
      <Paragraph position="0"> Currently the problem of ambiguity is solved by endowing the analyzer with a chart and employing the semantic density algorithm described in (Bryant, 2004). In the future probabilistic reasoning frameworks as proposed by (Narayanan and Jurafsky, 2005) in combination with ontology-based coherence measures as proposed by (Loos and Porzel, 2004) constitute promising approaches for handling problems of construal, whether it be on a pragmatic, semantic, syntactic or phonological level.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>