File Information
File: 05-lr/acl_arc_1_sum/cleansed_text/xml_by_section/metho/04/w04-0202_metho.xml
Size: 27,601 bytes
Last Modified: 2025-10-06 14:09:03
<?xml version="1.0" standalone="yes"?> <Paper uid="W04-0202"> <Title>Kaplan, J., Cooperative Responses from a Portable</Title> <Section position="3" start_page="1" end_page="1" type="metho"> <SectionTitle> 3 A typology of cooperative functions </SectionTitle> <Paragraph position="0"> The typology below clearly needs further testing, stabilization and confirmation by annotators. However, it settles the main lines of cooperative discourse structure.</Paragraph> <Section position="1" start_page="1" end_page="1" type="sub_section"> <SectionTitle> 3.1 Typology of corpora </SectionTitle> <Paragraph position="0"> To carry out our study and subsequent evaluations, we considered three typical sources of cooperative discourses: Frequently Asked Questions (FAQ), Forums and email question-answer pairs (EQAP), these latter obtained by sending ourselves emails to relevant services (e.g. for tourism: tourist offices, airlines, hotels). The initial study was carried out on 350 question-answer pairs. Note that in the tourism domain, FAQ are rather specific: they are not readymade, prototypical questions. They are rather unstructured sets of questions produced e.g. via email by standard users. From that point of view, they are of much interest to us.</Paragraph> <Paragraph position="1"> We have about 50% pairs coming from FAQ, 25% from Forums and 25% from EQAP. The domains considered are basically large-public applications: tourism (60%, our implementations being based on this application domain), health (22%), sport, shopping and education. In all these corpora, no user model is assumed, and there is no dialogue: QA pairs are isolated, with no context. This is basically the type of communication encountered when querying the Web. Our corpus is only composed of written texts, but these are rather informal, and quite close in style to spoken QA pairs.</Paragraph> <Paragraph position="2"> FAQ, Forum and EQAP cooperative responses share several similarities, but have also some differences. Forums have in general longer responses (up to half a page), whereas FAQ and EQAP are rather short (from 2 to 12 lines, in general). FAQ and Forums deal with quite general questions while EQAP are more personal. EQAP provided us with a very rich material since they allowed us to get responses to queries in which we have deliberately introduced various well identified errors and misconceptions. In order to have a better analysis of how humans react, we sent those questions to different, closely related organizations (e.g. sending the same ill-formed questions to several airlines). FAQ, Forums and EQAP also contain several forms of advertising, and metalinguistic parameters outlining e.g. their commercial dimensions.</Paragraph> <Paragraph position="3"> From the analysis of 350 of QA pairs, taking into account the formal pragmatics and artificial intelligence perspectives, we have identified the typology presented below, which defines the first version of COOPML.</Paragraph> </Section> <Section position="2" start_page="1" end_page="1" type="sub_section"> <SectionTitle> 3.2 Cooperative discourse functions </SectionTitle> <Paragraph position="0"> We structure cooperative responses in terms of co-operative functions, which are realized in responses by means of meaningful units (MU). An MU is the smallest unit we consider at this level; it conveys a minimal, but comprehensive and coherent fragment of information. In a response, MUs are connected by means of transition units (TU), which are introductory or inserted between meaningful units. TUs define the articulations of the cooperative discourse.</Paragraph> <Paragraph position="1"> In a cooperative discourse, we distinguish three types of MU: direct responses (DR), cooperative know-how (CSF) and units with a marginal usefulness (B) such as commentaries (BC), paraphrases (BP), advertising, useless explanations w.r.t. to the question. These may have a metalinguistic force (insistence, customer safety, etc) that we will not examine in this paper. DR are not cooperative by themselves, but they are studied here because they introduce cooperative statements. Let us now present a preliminary typology for DR and CSF, between parentheses are abbreviations used as XML labels.</Paragraph> <Paragraph position="2"> Direct responses (DR): are MUs corresponding to statements whose contents can be directly elaborated from texts, web pages, databases, etc., possibly via deduction, but not involving any reformulation of the original query. DR include the following main categories: * Simple responses (DS): consisting of yes/no forms, modals, figures, propositions in either affirmative or negative form, that directly respond the question.</Paragraph> <Paragraph position="3"> * Definitions, Descriptions (DD): usually text fragments defining or describing a concept, in response to questions e.g. of the form what is 'concept'?.</Paragraph> <Paragraph position="4"> * Procedures (DP): that describe how to realize something.</Paragraph> <Paragraph position="5"> * Causes, Consequences, Goals (DCC): that usually respond to questions in Why/ How?.</Paragraph> <Paragraph position="6"> * Comparisons and Evaluations (DC): that re- null spond to questions asking for comparisons or evaluations.</Paragraph> <Paragraph position="7"> This classification is closely related to a typology of questions defined in (Lehnert, 1978).</Paragraph> </Section> <Section position="3" start_page="1" end_page="1" type="sub_section"> <SectionTitle> Responses involving Cooperative Know-how </SectionTitle> <Paragraph position="0"> (CSF) are responses that go beyond direct answers in order to help the user when the question has no direct solution or when the question contains a misconception of some sort. These responses reflect various forms of know-how deployed by humans.</Paragraph> <Paragraph position="1"> We decompose them into two main classes: Response Elaboration (ER) and Additional Information (CR). The first class includes response units that propose alternative responses to the question whereas the latter contains a variety of complements of information, which are useful but not absolutely necessary. ER are in a large part inspired from specific research in Artificial Intelligence such as constraint relaxation and intensional calculus.</Paragraph> <Paragraph position="2"> Response elaboration (ER) includes the following MUs: * Corrective responses (CC): that explain why a question has no response when it contains a misconception or a false presupposition (formally, a domain integrity constraint or a factual knowledge violation, respectively), For example: Q5: a chalet in Corsica for 15 persons? has no solution, a possible response is: R5a: Chalets can accomodate a maximum of 10 persons in Corsica.</Paragraph> <Paragraph position="3"> * Responses by extension (CSFR): propose al null ternative solutions by relaxing a constraint in the original question. There are several forms of relaxations, reported in (Benamara et al.</Paragraph> <Paragraph position="4"> 2004a), which are more subtle than those developed in artificial intelligence. For example, we observed relaxation on cardinality, on sister concepts or on remote concepts with similar prominent properties, not studied in AI, where relaxation operates most of the time on the basis of ancestors.</Paragraph> <Paragraph position="5"> Response R5a above can then be followed by CSFRs of various forms such as: R5b: we can offer (1) two-close-by chalets for a total of 15 persons, or (2) another type of accomodation in Corsica: hotel or pension for 15 persons.</Paragraph> <Paragraph position="6"> Case (1) is a relaxation on cardinality (duplication of the resource) while (2) is a relaxation that refers to sisters of the concept chalet.</Paragraph> <Paragraph position="7"> * Intensional responses (CSFRI): tend to abstract over possibly long enumerations of extensional responses in order to provide a response at the best level of abstraction, which is not necessarily the highest. For example, Q6: How can I get to Geneva airport ? has the following response: null R6a: Taxis, most buses and all trains go to Geneva airport. This level is prefered to the more general but less informative response R6b: Most public transportations go to Geneva airport.</Paragraph> <Paragraph position="8"> * Indirect responses (CSFI): provide responses which are not direct w.r.t. the question (but which may have a direct response), e.g.: is your camping close to the highway?, can be indirectly, but cooperatively answered: yes, but that highway is quiet at night.. A direct response would have said, e.g.: yes, we are only 50 meters far from the highway, meaning that the camping is of an easy access.</Paragraph> <Paragraph position="9"> * Hypothetical responses (CSFH): include responses based on an hypothesis. Such responses are often related to incomplete questions, or questions which can only be partly be answered for various reasons such as lack of information, or vague information w.r.t the question focus. In this case, we have a QA pair of the form: Q7: Can I get discounts on train tickets ? R7: You can get a discount if you are less than 18 years old or more than 65, or if you are travelling during week-ends.</Paragraph> <Paragraph position="10"> * Clustered, case or comparative responses (CSFC): which answer various forms of questions e.g. with vague terms (e.g. expensive, far from the beach). For example, to Q8: is the hotel Royal expensive? it is answered: R8: for its category (3*) it is expensive, you can find 4* hotels at the same rate.</Paragraph> <Paragraph position="11"> The most frequent forms of responses are CSFR, CSFI, CSFC, CSFRI; the two others (CC and CSFH) are mainly found in email QA.</Paragraph> <Paragraph position="12"> Additional Information units (CR) contain the following cases: * precisions of various forms, that deepen the response (AF): this 'segment' or 'continuum' of forms ranges from minor precisions and generalizations to elaborated comments, as in Q9: Where can I buy a hiking trail map of Mount Pilat ? which has the response R9 that starts by an AF: R9: The parc published a 1:50 000 map with itineraries,... this map can be bought at bookshops....</Paragraph> <Paragraph position="13"> * restrictions (AR): restrict the scope of a response, e.g. by means of conditions: Q10: Do you refund tickets in case of a strike ? R10: yes, a financial compensation is possible provided that the railway union agrees....</Paragraph> <Paragraph position="14"> * warnings (AA): warn the questioner about possible problems, annoyances, dangers, etc.</Paragraph> <Paragraph position="15"> They may also underline the temporal versatility of the information, as it is often the case for touristic resources (for example, hotel or flight availability), * justifications (AJ): justify a negative, unexpected or partial response: Q11: CanIberefunded if I loose my rail pass ?, R11: No, the rail pass fare does not include any insurance against loss or robbery.</Paragraph> <Paragraph position="16"> * concessives (AC): introduce the possibility of e.g. exceptions or specific treatments: Children below 12 are not allowed to travel unaccompanied, however if a passenger is willing to take care about him....</Paragraph> <Paragraph position="17"> * suggestions - alternatives - counter-proposals (AS): this continuum of possibilities includes the proposition of alternatives, more or less marked, when the query has no answer, in particular via the above ER. Q12: Can I pay the hotel with a credit card?, R12: yes, but it is preferable to have cash with you: you'll get a much better exchange rate and no commission.</Paragraph> <Paragraph position="18"> The different MU have been designed with no overlap, it is however clear that there may have some forms of continuums between them. For example, CSFR, although more restricted, may be viewed as an AS, since an alternative, via relaxation, is proposed. We then would give preference to the CSF group over the CR, because they are more precise. null A response does not involve more, in general, than 3 to 4 meaningful units. Most are linearly organized, but some are also embedded. At the form level, response units of CSF (ER and CR) have in general one or a combination of the following forms: adverb or modal (RON), proposition (RP), enumeration (RE), sorted response (via e.g. scalar implicature) (RT), conditionals (RC) or case structure (RSC). These forms may have some overlap, e.g. RE and RT.</Paragraph> </Section> <Section position="4" start_page="1" end_page="1" type="sub_section"> <SectionTitle> 3.3 Annotating Cooperative Discourse: a few illustrations </SectionTitle> <Paragraph position="0"> Fig. 1 (next page) presents three examples annotated with COOPML.</Paragraph> </Section> <Section position="5" start_page="1" end_page="1" type="sub_section"> <SectionTitle> 3.4 Identifying cooperative response units </SectionTitle> <Paragraph position="0"> The question that arises at this stage is the existence of linguistic markers that allow for the identification of these response units. Besides these markers, there are also constraints on the organization of the cooperative discourse in meaningful units.</Paragraph> <Paragraph position="1"> These are essentially co-occurrence, incompatibility and precedence constraints. Finally, it is possible to elaborate heuristics that give indications on the most frequent combinations to improve MU automatic identification.</Paragraph> <Paragraph position="2"> In the following subsections we first present a typology for MU delimitation, then we explain how direct responses (DS) are identified, mainly, via the Discourse level: Q1: Can we buy drinking water on the Kilimandjaro ? R1: <DS>yes </DS>, <BP>drinking water can be bought </BP>, <CSP><AA>but fares are higher than in town, up to 2USD </AA>.<AR>It is however not allowed to bring much water from the city with you </AR></CSP>.</Paragraph> <Paragraph position="3"> Q2: Is there a cinema in Borme ? R2: <DS>No</DS>, <CSFR>the closest cinema is at Londes (8 kms) or at Hyeres (<AF>Cinema Olbia</AF>at 20 kms).</CSFR> Q3: How can I get to the Borme castle ? R3: <DS>You must take the GR90 from the old castle: <AF>walking distance: 30 minutes </AF>< /DS >. <AJ>There is no possibility to get there by car.</AJ> Form level: R2: <RON>No, </RON><RE><RT>The closest cinema is at Londes (8kms) or at Hyeres (cinema Olbia at 20 kms) </RT></RE>.</Paragraph> <Paragraph position="4"> domain ontology whose structure and contents is presented. We end the section by the linguistic marks that identify a number of additional information units (CR).</Paragraph> </Section> <Section position="6" start_page="1" end_page="1" type="sub_section"> <SectionTitle> 3.4.1 Typology of MU delimitators </SectionTitle> <Paragraph position="0"> Identifying meaningful response units consists in two tasks: exploring linguistic criteria associated with each form of cooperative response unit and finding the boundaries of each unit. Cooperative discourse being in general quite straightforward, it turns out that most units are well delimited naturally: about 70% of the units are single, complete sentences, ending by a dot. The others are either delimited by transition units TU such as connectors (about 20%) or by specific signs (e.g. end of enumerations, punctuation marks). Delimiting units is therefore in our perspective quite simple (it may not be so in e.g. oral QA or dialogues).</Paragraph> </Section> <Section position="7" start_page="1" end_page="1" type="sub_section"> <SectionTitle> 3.4.2 Identification of direct responses (DS) via </SectionTitle> <Paragraph position="0"> the domain ontology The identification (and the production) of a number of cooperative functions (e.g. relaxation, intensional responses, direct responses) rely heavily on ontological knowledge.</Paragraph> <Paragraph position="1"> Let us present first the characteristics of the ontology required in our approach. It is basically a conceptual ontology where nodes are associated with concept lexicalizations and essential properties. Each node is represented by the predicate : onto-node(concept, lex, properties) where concept has properties and lexicalisations lex. Most lexicalisations are entries in the lexicon (except for paraphrases), where morphological and grammatical aspects are described. For example, for hotel, we have (coded in Prolog): onto-node(hotel, [[hotel], [residence, hoteliere]], [night-rate, nb-of-rooms, facilities]) .</Paragraph> <Paragraph position="2"> There are several well-designed public domain ontologies on the net. Our ontology is a synthesis of two existing French ontologies, that we customized: TourinFrance (www.tourinfrance.net) and the bilingual (French and English) thesaurus of tourism and leisure activities (www.iztzg.hr/indokibiblioteka/THESAUR.PDF) which includes 2800 French terms. We manually integrated these ontologies in WEBCOOP (Benamara et al. 2004a) by removing concepts that are either too specific (i.e. too low level), like some basic aspects of ecology or rarely considered, as e.g. the economy of tourism. We also removed quite surprising classifications such as sanatorium under tourist accommodation. We finally reorganized some concept hierarchies, so that they 'look' more intuitive for a large public. Finally, we found that some hierarchies are a little bit odd, for example, we found at the same level accommodation capacity and holiday accommodation whereas, in our case, we consider that capacity is a property of the concept tourist accommodation.</Paragraph> <Paragraph position="3"> We have, at the moment, 1000 concepts in our tourism ontology which describe accommodation and transportation and a few other satellite elements (geography, health, immigration). Besides the traditional 'isa' relation, we also coded the 'part-of' relation. Synonymy is encoded via the list of lexicalizations. null Direct responses (DS) are essentially characterized by introductory markers like yes/no/this is possible and by the use of similar terms as those given in the question (55% of the cases) or by various lexicalizations of the question terms, studied in depth in (Benamara et al, 2004b). An obvious situation is when the response contains a subtype of the question focus: opening hours of the hotel - l'hotel vous acceuille 24h sur 24 (approx. hotel welcomes you round the clock). In terms of portability to other domains than tourism, note that the various terms used can be identified via the ontology: synonyms, sisters, subtypes.</Paragraph> <Paragraph position="4"> In this section, for space reasons, we explore only three typical CR: justifications (AJ), restrictions (AR) and warnings (AA). These MUs are characterized by markers which are general terms, domain independent for most of them. The study of these marks for French reveals that there is little marker overlap between units. Markers have been defined in a first stage from corpus analysis and then generalized to similar terms in order to have a larger basis for evaluation. We also used, to a limited extend, a bootstrapping technique to get more data (Ravinchandran and Hovy 2002), a method that starts by an unambiguous set of anchors (often arguments of a relational term) for a target sense. Searching text fragments on the Web based on these anchors then produces a number of ways of relating these anchors. null Let us now characterize linguistic markers for each of these categories: Restrictions (AR) are an important unit in cooperative discourse. There is a quite large literature in linguistics about the expression of restrictions. In cooperative discourse, the expression of restrictions is realized quite straightforwardly by a small number of classes of terms: (a) restrictive locutions: sous r'eserve que, `a l'exception de, il n'est pas autoris'e de, toutefois, etc. (provided that), (b) the negative form ne ... que that is typical of restrictions, is very frequently used (c) restrictive modals: doit obligatoirement, imp'erativement, n'ecessairement (must obligatorily), null (d) quantification with a restrictive interpretation: seul, pas tous, au maximum (only, not all).</Paragraph> <Paragraph position="5"> Justifications (AJ) is also an important meaningful unit, it has however a little bit fuzzy scope. Marks are not very clearcut. Among them, we have: (a) marks expressing causality, mainly connectors such as: car, parce que, en raison de, (b) marks expressing, via other forms of negation than in AR, the impossibility to give a positive response, or marks 'justifying' the response: il n'y a pas, il n'existe pas, en effet (because, there is no, indeed).</Paragraph> <Paragraph position="6"> Warnings (AA) can quite clearly be identified by means of: (a) verbal expressions: sachez que, veuillez `ane pas, mieux vaut 'eviter, n'oubliez pas, attention `a, etc. (note that, do not forget, etc.), (b) expressions or temporal morphological marks that indicate that data is sensitive to time and may be true only at some point: mise `a jour, changements fr'equents, etc. (frequent updates), (c) a few other expressions such as: il n'existe pas, mais (but) ... + comparative form.</Paragraph> <Paragraph position="7"> Except for the identification of DS, which require quite a lot of ontological resources, marks identified for the other MU studied here are quite general. Portability of these marks to other domains and possibly to other languages should be a reasonably feasible challenge.</Paragraph> <Paragraph position="8"> The response elaboration part (ER) is more constrained in terms of marks, because of the logical procedures that are related to. For example, the CSFR, dealing with constraint relaxation, involves the use of sister, daughter and sometimes parent nodes of the focus, and often proposes at least 2 choices. It is in general associated with a negative direct response, or an explanation why no response can be found. It also also contains some fixed marks that indicate a change of concept, such as another type of. This is easily visible in the pair Q2-R2 (section 3.3) with the mark: the closests.</Paragraph> <Paragraph position="9"> A few constraints or preferences can be formulated on the organization of meaningful units, these may be somewhat flexible, because cooperative discourse may have a wide range of forms: (a) coocurrence: any DR can co-occur with an AS, AF, AR, AA or AJ, (b) precedence: any DR precedes any (unmarked) AA, AR, AC, ACP, B, or any sequence DS-BP. Any CC precedes any CSFR, CSFH or CSFRI, (c) incompatibility: DS + DP, CSFR + CSFI, CSFC + CSFH. Furthermore CR cannot appear alone.</Paragraph> <Paragraph position="10"> Frequent pairs are quite numerous, here are the most typical ones: DS + P, DS + AR, CC + CSFR or CSFH or CSFRI, DS + AJ, DS(negative) + AJ + AS, DS + AF, DS(negative) + CSFR. These can be considered in priority in case of ambiguities.</Paragraph> </Section> <Section position="8" start_page="1" end_page="1" type="sub_section"> <SectionTitle> 3.5 Evaluation by annotators </SectionTitle> <Paragraph position="0"> At this stage, it is necessary to have evaluated by human annotators how clear, well-delimited and easy to use this classification is. We do not have yet precise results, but it is clear that judgments may vary from one annotator to another. This is not only due to the generic character of our definitions, but also to the existence of continuums between categories, and to the interpretation of responses that may vary depending on context, profile and culture of annotators. null An experiment carried out on three independent subjects (annotation task followed by a discussion of the results) reveals that there is a clear consensus of 80% on the annotations we did ourselves.</Paragraph> <Paragraph position="1"> The other 20% reflect interpretation variations, in general highly contextual. These 20% are almost the same cases for the three subjects. In particular, at the level of additional information (CR), we observed some differences in judgement in particular between restrictions (AR) and warnings (AA), and a few others between CSFH and CSFC whose differences may sometimes be only superficial (presentation of the arguments of the response).</Paragraph> <Paragraph position="2"> 3.6 Evaluation of prototype: a first experiment We can now evaluate the accuracy of the linguistic marks given above. For that purpose, we designed a programme in Prolog (for fast prototyping) that uses: (1) the domain lexicon and ontology, to have access e.g. to term lexicalizations and morphology, and (2) a set of 'local' grammars that implement the different marks. Since these marks involve lexical and morphological variations, negation, and some long-distance dependencies, grammars are a good solution.</Paragraph> <Paragraph position="3"> Tests were carried out on a new corpus, essentially from airlines FAQ. 134 QA pairs have been selected from this corpus containing some form of cooperativity. The annotation of this corpus is automatic, while the evaluation of the results is manual and is carried out in parallel by both ourselves and by an external professional evaluator. These 134 QA pairs contain a total of 237 MU, therefore an average of 1.76 MU per response. Most responses have 2 MU, the maximum observed being 4. Surprisingly, out of the 134 pairs, only 108 contain direct responses followed by various CSF, the other 16 only contain cooperative know-how responses (CSF), without any direct response part.</Paragraph> <Paragraph position="4"> Evaluation results, although carried out on a relatively small set of QA pairs, give good indications on the accuracy of the linguistic marks, and also on the typology of the different MU. We consider here the MU: DS, AJ, AR, AA, as characterized above: A: number of MU annotated correctly for that category, B: MU not annotated (no decision made), C: incorrect annotation.</Paragraph> <Paragraph position="5"> MU boundaries have been correctly identified in 88% of the cases, they are mostly related to punctuation marks.</Paragraph> <Paragraph position="6"> There are obviously a few delicate cases where annotation is difficult if not impossible. First, we observed a few discontinuities: an MU can be fragmented. In that case, it is necessary to add an index to the tag so that the different fragments can be unambiguously related, as in: Q: What is the deadline for an internet reservation? R: <DRindex=1> In the case of an electronic ticket, you can reserve up to 24h prior to departure </DR>.<B>You just need to show up at the registration desk < /B > . < DR index =1> In the case of a traditional ticket ... </DR>.</Paragraph> <Paragraph position="7"> The index=1 allows to tie the two fragments of the enumeration.</Paragraph> <Paragraph position="8"> In a number of cases the direct response part is rather indirect, making its identification via the means presented above quite delicate: Q: I forgot to note my reservation number, how can I get it? R: A confirmation email has been sent to you as soon as the reservation has been finalized....To identify this portion of the response as a DR, it is necessary to infer that the email is a potential container for a reservation number.</Paragraph> </Section> </Section> class="xml-element"></Paper>