File Information
File: 05-lr/acl_arc_1_sum/cleansed_text/xml_by_section/metho/04/p04-1004_metho.xml
Size: 27,637 bytes
Last Modified: 2025-10-06 14:08:52
<?xml version="1.0" standalone="yes"?> <Paper uid="P04-1004"> <Title>Analysis of Mixed Natural and Symbolic Language Input in Mathematical Dialogs</Title> <Section position="4" start_page="0" end_page="0" type="metho"> <SectionTitle> 3 Linguistic data </SectionTitle> <Paragraph position="0"> In this section, we first briefly describe the corpus collection experiment and then present the common language phenomena found in the corpus.</Paragraph> <Section position="1" start_page="0" end_page="0" type="sub_section"> <SectionTitle> 3.1 Corpus collection </SectionTitle> <Paragraph position="0"> 24 subjects with varying educational background and little to fair prior mathematical knowledge participated in a Wizard-of-Oz experiment (Benzm&quot;uller et al., 2003b). In the tutoring session, they were asked to prove 3 theorems2:</Paragraph> <Paragraph position="2"> To encourage dialog with the system, the subjects were instructed to enter proof steps, rather than complete proofs at once. Both the subjects and the tutor were free in formulating their turns. Buttons were available in the interface for inserting mathematical symbols, while literals were typed on the keyboard. The dialogs were typed in German.</Paragraph> <Paragraph position="3"> The collected corpus consists of 66 dialog logfiles, containing on average 12 turns. The total number of sentences is 1115, of which 393 are student sentences. The students' turns consisted on average of 1 sentence, the tutor's of 2. More details on the corpus itself and annotation efforts that guide the development of the system components can be found in (Wolska et al., 2004).</Paragraph> <Paragraph position="4"> 2a51 stands for set complement and a52 for power set.</Paragraph> </Section> <Section position="2" start_page="0" end_page="0" type="sub_section"> <SectionTitle> 3.2 Language phenomena </SectionTitle> <Paragraph position="0"> To indicate the overall complexity of input understanding in our setting, we present an overview of common language phenomena in our dialogs.3 In the remainder of this paper, we then concentrate on the issue of interleaved natural language and mathematical expressions, and present an approach to processing this type of input.</Paragraph> <Paragraph position="1"> Interleaved natural language and formulae Mathematical language, often semi-formal, is interleaved with natural language informally verbalizing proof steps. In particular, mathematical expressions (or parts thereof) may lie within the scope of quantifiers or negation expressed in natural language: For parsing, this means that the mathematical content has to be identified before it is interpreted within the utterance.</Paragraph> <Paragraph position="2"> Imprecise or informal naming Domain relations and concepts are described informally using imprecise and/or ambiguous expressions.</Paragraph> <Paragraph position="3"> A enthaelt B [A contains B] A muss in B sein [A must be in B] where contain and be in can express the domain relation of either subset or element; where be outside of and be different are informal descriptions of the empty intersection of sets. To handle imprecision and informality, we constructed an ontological knowledge base containing domain-specific interpretations of the predicates (Horacek and Wolska, 2004).</Paragraph> <Paragraph position="4"> Discourse deixis Anaphoric expressions refer deictically to pieces of discourse: der obere Ausdruck [the above term] der letzte Satz [the last sentence] Folgerung aus dem Obigen [conclusion from the above] aus der regel in der zweiten Zeile [from the rule in the second line] 3As the tutor was also free in wording his turns, we include observations from both student and tutor language behavior. In the presented examples, we reproduce the original spelling. In our domain, this class of referring expressions also includes references to structural parts of terms and formulae such as &quot;the left side&quot; or &quot;the inner parenthesis&quot; which are incomplete specifications: the former refers to a part of an equation, the latter, metonymic, to an expression enclosed in parenthesis. Moreover, these expressions require discourse referents for the sub-parts of mathematical expressions to be available.</Paragraph> <Paragraph position="5"> Generic vs. specific reference Generic and specific references can appear within one utterance: Potenzmenge enthaelt alle Teilmengen, also auch (Aa73 B) [A power set contains all subsets, hence also(Aa73 B)] where &quot;a power set&quot; is a generic reference, whereas &quot;a4a32a12a26a8 &quot; is a specific reference to a subset of a specific instance of a power set introduced earlier. Co-reference4 Co-reference phenomena specific to informal mathematical discourse involve (parts of) mathematical expressions within text.</Paragraph> <Paragraph position="6"> Da, wenn a0a2a1a5a53a13a54a57a56a59a58a4a3a42a60 sein soll, a0a2a1 Element vona54a57a56a58a5a3 a60 sein muss. Und wenna58a2a6 a53a13a54a57a56a7a0a2a8a60 sein soll, muss a9a11a10a12a6 auch Element vona54a57a56a7a0 a8a60 sein.</Paragraph> <Paragraph position="7"> [Because if it should be that a0a2a1a16a53a13a54a57a56a58a5a3 a60, a0a2a1 must be an element ofa54a57a56a59a58a5a3a42a60. And if it should be thata58 a6 a53a13a54a57a56a7a0 a8a60, it must be an element ofa54a57a56a7a0 a8a60 as well.] Entities denoted with the same literals may or may not co-refer:</Paragraph> <Paragraph position="9"> Informal descriptions of proof-step actions Sometimes, &quot;actions&quot; involving terms, formulae or parts thereof are verbalized before the appropriate formal operation is performed: Wende zweimal die DeMorgan-Regel an [I'm applying DeMorgan rule twice] damit kann ich den oberen Ausdruck wie folgt schreiben:. . . [given this I can write the upper term as follows:. . . ] The meaning of the &quot;action verbs&quot; is needed for the interpretation of the intended proof-step.</Paragraph> <Paragraph position="10"> Metonymy Metonymic expressions are used to refer to structural sub-parts of formulae, resulting in predicate structures acceptable informally, yet incompatible in terms of selection restrictions.</Paragraph> <Paragraph position="11"> Dann gilt fuer die linke Seite, wenn a21a23a22a25a24a27a26a29a28a31a30a33a32a35a34a36a24a7a26a29a22a31a21a37a32a38a28a25a24a27a30a39a22a40a21a37a32 , der Begriff A a28 B dann ja schon dadrin und ist somit auch Element davon [Then for the left hand side it holds that..., the term A a28 B is already there, and so an element of it] 4To indicate co-referential entities, we inserted the indices which are not present in the dialog logfiles.</Paragraph> <Paragraph position="12"> where the predicate hold, in this domain, normally takes an argument of sort CONST, TERM or FOR-MULA, rather than LOCATION; de morgan regel 2 auf beide komplemente angewendet [de morgan rule 2 applied to both complements] where the predicate apply takes two arguments: one of sort RULE and the other of sort TERM or FOR-MULA, rather than OPERATION ON SETS.</Paragraph> <Paragraph position="13"> In the next section, we present our approach to a uniform analysis of input that consists of a mixture of natural language and mathematical expressions.</Paragraph> <Paragraph position="14"> 4 Uniform input analysis strategy The task of input interpretation is two-fold. Firstly, it is to construct a representation of the utterance's linguistic meaning. Secondly, it is to identify and separate within the utterance: (i) parts which constitute meta-communication with the tutor, e.g.: Ich habe die Aufgabenstellung nicht verstanden.</Paragraph> <Paragraph position="15"> [I don't understand what the task is.] (ii) parts which convey domain knowledge that should be verified by a domain reasoner; for example, the entire utterance a41 a24a42a24a27a26a29a22a40a30a33a32a43a32 ist laut deMorgan-1 a41 a24a7a26a44a32a38a28 a41 a24a27a30a2a32 [. . . is, according to deMorgan-1,. . . ] can be evaluated; on the other hand, the domain reasoner's knowledge base does not contain appropriate representations to evaluate the correctness of using, e.g., the focusing particle &quot;also&quot;, as in: Wenn A = B, dann ist A auch a45 a41 a24a27a30a2a32 und B a45 a41 a24a27a26a44a32 . [If A = B, then A is also a45 a41 a24a27a30a2a32 and B a45 a41 a24a27a26a44a32 .] Our goal is to provide a uniform analysis of inputs of varying degrees of verbalization. This is achieved by the use of one grammar that is capable of analyzing utterances that contain both natural language and mathematical expressions. Syntactic categories corresponding to mathematical expressions are treated in the same way as those of linguistic lexical entries: they are part of the deep analysis, enter into dependency relations and take on semantic roles. The analysis proceeds in 2 stages: 1. After standard pre-processing,5 mathematical expressions are identified, analyzed, categorized, and substituted with default lexicon entries encoded in the grammar (Section 4.1).</Paragraph> <Paragraph position="16"> 5Standard pre-processing includes sentence and word tokenization, (spelling correction and) morphological analysis, part-of-speech tagging.</Paragraph> <Paragraph position="18"> 2. Next, the input is syntactically parsed, and a representation of its linguistic meaning is constructed compositionally along with the parse (Section 4.2).</Paragraph> <Paragraph position="19"> The obtained linguistic meaning representation is subsequently merged with discourse context and interpreted by consulting a semantic lexicon of the domain and a domain-specific knowledge base (Section 4.3).</Paragraph> <Paragraph position="20"> If the syntactic parser fails to produce an analysis, a shallow chunk parser and keyword-based rules are used to attempt partial analysis and build a partial representation of the predicate-argument structure. In the next sections, we present the procedure of constructing the linguistic meaning of syntactically well-formed utterances.</Paragraph> </Section> <Section position="3" start_page="0" end_page="0" type="sub_section"> <SectionTitle> 4.1 Parsing mathematical expressions </SectionTitle> <Paragraph position="0"> The task of the mathematical expression parser is to identify mathematical expressions. The identified mathematical expressions are subsequently verified as to syntactic validity and categorized.</Paragraph> </Section> <Section position="4" start_page="0" end_page="0" type="sub_section"> <SectionTitle> Implementation Identification of mathematical </SectionTitle> <Paragraph position="0"> expressions within word-tokenized text is performed using simple indicators: single character tokens (with the characters a29 and a0 standing for power set and set complement respectively), mathematical symbol unicodes, and new-line characters.</Paragraph> <Paragraph position="1"> The tagger converts the infix notation used in the input into an expression tree from which the following information is available: surface sub-structure (e.g., &quot;left side&quot; of an expression, list of sub-expressions, list of bracketed sub-expressions) and expression type based on the top level operator (e.g., CONST, TERM, FORMULA 0 FORMULA (formula missing left argument), etc.).</Paragraph> <Paragraph position="2"> For example, the expression a0a2a1a3a1a5a4 a6a30a8a11a10 a12a7a1a16a15 a6 a17a18a10a3a10a20a19a21a1a5a0a2a1a5a4 a6a18a8 a10a25a12 a0a2a1a16a15 a6 a17a26a10 ) is represented by the formula tree in Fig. 1. The bracket subscripts indicate the operators heading sub-formulae enclosed in parenthesis. Given the expression's top node operator, =, the expression is of type formula, its &quot;left side&quot; is the expressiona0a2a1a3a1a5a4 a6 a8a11a10a12a70a1a16a15a24a6a20a17a26a10a3a10 , the list of bracketed sub-expressions includes: Aa6 B, Ca6 D, a1a5a4a32a6a26a8a11a10a25a12a34a1a16a15 a6a26a17a18a10 , etc.</Paragraph> <Paragraph position="3"> Evaluation We have conducted a preliminary evaluation of the mathematical expression parser.</Paragraph> <Paragraph position="4"> Both the student and tutor turns were included to provide more data for the evaluation. Of the 890 mathematical expressions found in the corpus (432 in the student and 458 in the tutor turns), only 9 were incorrectly recognized. The following classes of errors were detected:6 [The same holds with . . . ] The examples in (1) and (2) have to do with parentheses. In (1), the student actually omitted them. The remedy in such cases is to ask the student to correct the input. In (2), on the other hand, no parentheses are missing, but they are ambiguous between mathematical brackets and parenthetical statement markers. The parser mistakenly included one of the parentheses with the mathematical expressions, thereby introducing an error. We could include a list of mathematical operations allowed to be verbalized, in order to include the logical connective in (2a) in the tagged formula. But (2b) shows that this simple solution would not remedy the problem overall, as there is no pattern as to the amount and type of linguistic material accompanying the formulae in parenthesis. We are presently working on ways to identify the two uses of parentheses in a pre-processing step. In (3) the error is caused by a non-standard character, &quot;?&quot;, found in the formula. In (4) the student omitted punctuation causing the character &quot;D&quot; to be interpreted as a non-standard literal for naming an operation on sets.</Paragraph> </Section> <Section position="5" start_page="0" end_page="0" type="sub_section"> <SectionTitle> 4.2 Deep analysis </SectionTitle> <Paragraph position="0"> The task of the deep parser is to produce a domain-independent linguistic meaning representation of syntactically well-formed sentences and fragments.</Paragraph> <Paragraph position="1"> By linguistic meaning (LM), we understand the dependency-based deep semantics in the sense of the Prague School notion of sentence meaning as employed in the Functional Generative Description 6Incorrect tagging is shown along with the correct result below it, following an arrow.</Paragraph> <Paragraph position="2"> (FGD) (Sgall et al., 1986; Kruijff, 2001). It represents the literal meaning of the utterance rather than a domain-specific interpretation.7 In FGD, the central frame unit of a sentence/clause is the head verb which specifies the tectogrammatical relations (TRs) of its dependents (participants). Further distinction is drawn into inner participants, such as Actor, Patient, Addressee, and free modifications, such as Location, Means, Direction. Using TRs rather than surface grammatical roles provides a generalized view of the correlations between domain-specific content and its linguistic realization. null We use a simplified set of TRs based on (HajiVcov'a et al., 2000). One reason for simplification is to distinguish which relations are to be understood metaphorically given the domain sub-language. In order to allow for ambiguity in the recognition of TRs, we organize them hierarchically into a taxonomy. The most commonly occurring relations in our context, aside from the inner participant roles of Actor and Patient, are Cause, Condition, and Result-Conclusion (which coincide with the rhetorical relations in the argumentative structure of the proof), for example: Other commonly found TRs include Norm-Criterion, e.g.</Paragraph> <Paragraph position="4"> [. . . equals, according to De Morgan rule1, . . . ] We group other relations into sets of HasProperty, GeneralRelation (for adjectival and clausal modification), and Other (a catch-all category), for example: null dann muessen alla A und B [in C]a6 PROP-LOC fers in coverage: while it does operate on the level of deep semantic roles, such aspects of meaning as the scope of quantifiers or interpretation of plurals, synonymy, or ambiguity are not resolved.</Paragraph> <Paragraph position="5"> where PROP-LOC denotes the HasProperty relation of type Location, GENREL is a general relation as in complementation, and PROP-FROM is a HasProperty relation of type Direction-From or From-Source. More details on the investigation into tectogrammatical relations that build up linguistic meaning of informal mathematical text can be found in (Wolska and Kruijff-Korbayov'a, 2004a).</Paragraph> <Paragraph position="6"> Implementation The syntactic analysis is performed using openCCG8, an open source parser</Paragraph> </Section> <Section position="6" start_page="0" end_page="0" type="sub_section"> <SectionTitle> for Multi-Modal Combinatory Categorial Gram- </SectionTitle> <Paragraph position="0"> mar (MMCCG). MMCCG is a lexicalist grammar formalism in which application of combinatory rules is controlled though context-sensitive specification of modes on slashes (Baldridge and Kruijff, 2003). The linguistic meaning, built in parallel with the syntax, is represented using Hybrid Logic Dependency Semantics (HLDS), a hybrid logic representation which allows a compositional, unification-based construction of HLDS terms with CCG (Baldridge and Kruijff, 2002). An HLDS term is a relational structure where dependency relations between heads and dependents are encoded as modal relations. The syntactic categories for a lexical entry FORMULA, corresponding to mathematical expressions of type &quot;formula&quot;, are a9 , a10 a29 , and a10 . For example, in one of the readings of &quot;B enthaelt a27a34a4 &quot;, &quot;enthaelt&quot; represents the meaning contain taking dependents in the relations Actor and Patient, shown schematically in Fig. 2.</Paragraph> <Paragraph position="1"> utterance &quot;B enthaelt a11 a27 a4 &quot; [B contains a11 a27a30a4 ]. FORMULA represents the default lexical entry for identified mathematical expressions categorized as &quot;formula&quot; (cf. Section 4.1). The LM is represented by the following HLDS term:</Paragraph> <Paragraph position="3"> where h1 is the state where the proposition contain is true, and the nominals f1 and f2 represent dependents of the head contain, which stand in the tectogrammatical relations Actor and Patient, respectively. null It is possible to refer to the structural sub-parts of the FORMULA type expressions, as formula sub-parts are identified by the tagger, and discourse ref- null erents are created for them and stored with the discourse model.</Paragraph> <Paragraph position="4"> We represent the discourse model within the same framework of hybrid modal logic. Nominals of the hybrid logic object language are atomic formulae that constitute a pointing device to a particular place in a model where they are true. The satisfaction operator, @, allows to evaluate a formula at the point in the model given by a nominal (e.g. the formula @a0 a1 evaluates a1 at the point i). For discourse modeling, we adopt the hybrid logic formalization of the DRT notions in (Kruijff, 2001; Kruijff and Kruijff-Korbayov'a, 2001). Within this formalism, nominals are interpreted as discourse referents that are bound to propositions through the satisfaction operator. In the example above, f1 and f2 represent discourse referents for FORMULA:B and FOR-MULA:a11 a27a49a4 , respectively. More technical details on the formalism can be found in the aforementioned publications.</Paragraph> </Section> <Section position="7" start_page="0" end_page="0" type="sub_section"> <SectionTitle> 4.3 Domain interpretation </SectionTitle> <Paragraph position="0"> The linguistic meaning representations obtained from the parser are interpreted with respect to the domain. We are constructing a domain ontology that reflects the domain reasoner's knowledge base, and is augmented to allow resolution of ambiguities introduced by natural language. For example, the previously mentioned predicate contain represents the semantic relation of Containment which, in the domain of naive set theory, is ambiguous between the domain relations ELEMENT, SUBSET, and PROPER SUBSET. The specializations of the ambiguous semantic relations are encoded in the ontology, while a semantic lexicon provides interpretations of the predicates. At the domain interpretation stage, the semantic lexicon is consulted to translate the tectogrammatical frames of the predicates into the semantic relations represented in the domain ontology. More details on the lexical-semantic stage of interpretation can be found in (Wolska and Kruijff-Korbayov'a, 2004b), and more details on the domain ontology are presented in (Horacek and Wolska, 2004).</Paragraph> <Paragraph position="1"> For example, for the predicate contain, the lexicon contains the following facts: ['the Containment relation involves a predicate contain and its Actor and Patient dependents, where the Actor and Patient are the container and containee parameters respectively'] Translation rules that consult the ontology expand the meaning of the predicates to all their alternative domain-specific interpretations preserving argument structure.</Paragraph> <Paragraph position="2"> As it is in the capacity of neither sentence-level nor discourse-level analysis to evaluate the correctness of the alternative interpretations, this task is delegated to the Proof Manager (PM). The task of the PM is to: (A) communicate directly with the theorem prover;9 (B) build and maintain a representation of the proof constructed by the student;10 (C) check type compatibility of proof-relevant entities introduced as new in discourse; (D) check consistency and validity of each of the interpretations constructed by the analysis module, with the proof context; (E) evaluate the proof-relevant part of the utterance with respect to completeness, accuracy, and relevance.</Paragraph> </Section> </Section> <Section position="5" start_page="0" end_page="0" type="metho"> <SectionTitle> 5 Example analysis </SectionTitle> <Paragraph position="0"> In this section, we illustrate the mechanics of the approach on the following examples.</Paragraph> <Paragraph position="1"> (1) B enthaelt kein a19 a20 a26 [B contains no a19 a20 a26 ] (2) Aa28 Ba20a24a38 Aa28 Ba39 (3) A enthaelt keinesfalls Elemente, die in B sind. [A contains no elements that are also in B] Example (1) shows the tight interaction of natural language and mathematical formulae. The intended reading of the scope of negation is over a part of the formula following it, rather than the whole formula. The analysis proceeds as follows.</Paragraph> <Paragraph position="2"> The formula tagger first identifies the formula a40 x a27 Aa41 and substitutes it with the generic entry FORMULA represented in the lexicon. If there was no prior discourse entity for &quot;B&quot; to verify its type, the type is ambiguous between CONST, TERM, and FORMULA.11 The sentence is assigned four alterna- null tive readings: (i) &quot;CONST contains no FORMULA&quot;, (ii) &quot;TERM contains no FORMULA&quot;, (iii) &quot;FORMULA contains no FORMULA&quot;, (iv) &quot;CONST contains no CONST 0 FORMULA&quot;. The last reading is obtained by partitioning an entity of type FORMULA in meaningful ways, taking into account possible interaction with preceding modifiers. Here, given the quantifier &quot;no&quot;, the expression a40 xa27 Aa41 has been split into its surface parts 9We are using a version of a42 MEGA adapted for assertionlevel proving (Vo et al., 2003).</Paragraph> <Paragraph position="3"> 10The discourse content representation is separated from the proof representation, however, the corresponding entities must be co-indexed in both.</Paragraph> <Paragraph position="4"> 11In prior discourse, there may have been an assignment B := a43 , where a43 is a formula, in which case, B would be known from discourse context to be of type FORMULA (similarly for term assignment); by CONST we mean a set or element variable such as A, x denoting a set A or an element x respectively. utterance &quot;B enthaelt kein a40a8a7a11a10a9 a7a27 a4 a9 a41 &quot; [B contains no a40a11a7a11a10a9 a7a27a14a4 a9 a41 ]. as follows: a40 [x][a27 A]a41 .12 [x] has been substituted with a generic lexical entry CONST, and [a27 A] with a symbolic entry for a formula missing its left argument (cf. Section 4.1).</Paragraph> <Paragraph position="5"> The readings (i) and (ii) are rejected because of ['B contains no x such that x is an element of A'] Next, the semantic lexicon is consulted to translate these readings into their domain interpretations. The relevant lexical semantic entries were presented in Section 4.3. Using the linguistic meaning, the semantic lexicon, and the ontology, we obtain four interpretations paraphrased below: -- for &quot;FORMULA contains no FORMULA&quot;: (1.1) 'it is not the case that a6 PATa7 , the formula, xa20 A, is a subformula of a6 ACTa7 , the formula B'; -- for &quot;CONST contains no CONST 0 FORMULA&quot;: 12There are other ways of constituent partitioning of the formula at the top level operator to separate the operator and its arguments: a12 [x][a13 ][A]a14 and a12 [xa13 ][A]a14 . Each of the partitions obtains its appropriate type corresponding to a lexical entry available in the grammar (e.g., the [xa13 ] chunk is of type FORMULA 0 for a formula missing its right argument). Not all the readings, however, compose to form a syntactically and semantically valid parse of the given sentence.</Paragraph> <Paragraph position="6"> 13Irrelevant parts of the meaning representation are omitted; glosses of the hybrid formulae are provided.</Paragraph> <Paragraph position="7"> and xa20 A'.</Paragraph> <Paragraph position="8"> The interpretation (1.1) is verified in the discourse context with information on structural parts of the discourse entity &quot;B&quot; of type formula, while (1.2a-c) are translated into messages to the PM and passed on for evaluation in the proof context.</Paragraph> <Paragraph position="9"> Example (2) contains one mathematical formula.</Paragraph> <Paragraph position="10"> Such utterances are the simplest to analyze: The formulae identified by the mathematical expression tagger are passed directly to the PM.</Paragraph> <Paragraph position="11"> Example (3) shows an utterance with domain-relevant content fully linguistically verbalized. The analysis of fully verbalized utterances proceeds similarly to the first example: the mathematical expressions are substituted with the appropriate generic lexical entries (here, &quot;A&quot; and &quot;B&quot; are substituted with their three possible alternative readings: CONST, TERM, and FORMULA, yielding several readings &quot;CONST contains no elements that are also in CONST&quot;, &quot;TERM contains no elements that are also in TERM&quot;, etc.). Next, the sentence is analyzed by the grammar. The semantic roles of Actor and Patient associated with the verb &quot;contain&quot; are taken by &quot;A&quot; and &quot;elements&quot; respectively; quantifier &quot;no&quot; is in the relation Restrictor with &quot;A&quot;; the relative clause is in the GeneralRelation with &quot;elements&quot;, etc. The linguistic meaning of the utterance in example (3) is shown in Fig. 5. Then, the semantic lexicon and the ontology are consulted to translate the linguistic meaning into its domain-specific interpretations, which are in this case very similar to the ones of example (1).</Paragraph> </Section> class="xml-element"></Paper>