<?xml version="1.0" standalone="yes"?>
<Paper uid="W06-1415">
  <Title>Generating Intelligent Numerical Answers in a Question-Answering System</Title>
  <Section position="5" start_page="103" end_page="103" type="metho">
    <SectionTitle>
2 On numerical data
</SectionTitle>
    <Paragraph position="0"> We focus on the integration of numerical data for the generation of natural language cooperative numerical answers. We first present some related work on generation from numerical data sets.</Paragraph>
    <Paragraph position="1"> Then we propose a model for the generation of co-operative numerical answers.</Paragraph>
  </Section>
  <Section position="6" start_page="103" end_page="104" type="metho">
    <SectionTitle>
2.1 Related work
</SectionTitle>
    <Paragraph position="0"> The generation of summaries from numerical data has been developed in some NLG systems. For example, the system ANA (Kukich, 1983) generates stock market reports by computing fluctuations for a day. FoG (Goldberg et al, 1994) produces weather forecasts from forecast data. More recently, StockReporter (Dale, 2003) was developed to generate summaries describing how a stock performs over a period. Yu et al. (2005) propose a system which generates summaries of sensor data from gas turbines.</Paragraph>
    <Paragraph position="1"> Those systems have input data analysis components which are more or less efficient and describe numerical time-series data. In the framework of QA systems, there are other major problems that the previous systems do not deal with. When a numerical question is submitted to a QA system, a set of numerical data is extracted from the Web.</Paragraph>
    <Paragraph position="2"> Then, the goal is not to describe the whole data set but to find an appropriate answer, dealing with the user expectations (for example, contraints in the question) or data inconsistencies. Another important point is the analysis of numerical input data in order to identify causes (besides time) of variation.</Paragraph>
    <Section position="1" start_page="103" end_page="103" type="sub_section">
      <SectionTitle>
2.2 A typology of numerical answers
</SectionTitle>
      <Paragraph position="0"> Our challenge is to develop a formal framework for the integration of numerical data extracted from Web pages in order to produce cooperative numerical answers.</Paragraph>
      <Paragraph position="1"> To define the different types of numerical answers, we collected a set of 80 question-answer pairs about prices, quantities, age, time, weight, temperature, speed and distance. The goal is to identify for each question-answer pair why extracted numerical values are different (is this an inconsistency? an evolution?).</Paragraph>
      <Paragraph position="2"> A numerical question may accept several answers when numerical values vary according to some criteria. Let us consider the following examples.</Paragraph>
      <Paragraph position="3"> Example 1 : How many inhabitants are there in France?  - Population census in France (1999): 60184186. - 61.7: number of inhabitants in France in 2004. Example 2 : What is the average age of marriage of women in 2004? - In Iran, the average age of marriage of women was 21 years in 2004.</Paragraph>
      <Paragraph position="4"> - In 2004, Moroccan women get married at the age of 27.</Paragraph>
      <Paragraph position="5"> Example 3 : At what temperature should I serve wine? - Red wine must be served at room temperature. - Champagne: between 8 and 10 @ C.</Paragraph>
      <Paragraph position="6"> - White wine: between 8 and 11 @ C.</Paragraph>
      <Paragraph position="7">  The corpus analysis allows us to identify 3 main variation criteria, namely time (ex.1), place (ex.2) and restriction (ex.3: restriction on the focus, for example: Champagne/wine). These criteria can be combined: some numerical values vary according to time and place, to time and restrictions, etc. (for example, the average age of marriage vary according to time, place and restrictions on men/women).</Paragraph>
    </Section>
    <Section position="2" start_page="103" end_page="104" type="sub_section">
      <SectionTitle>
2.3 A model for cooperative numerical
answer generation
</SectionTitle>
      <Paragraph position="0"> The system has to generate an answer from a set of numerical data. In order to identify the different problems, let us consider the following example : What is the average age of marriage in France? - In 1972, the average age of marriage was 24.5 for men and 22.4 for women. In 2005, it is 30 for men and 28 for women.</Paragraph>
      <Paragraph position="1"> - The average age of marriage in France increased from 24.5 to 26.9 for women and from 26.5 to 29 for men between 1986 and 1995.</Paragraph>
      <Paragraph position="2">  This set of potential answers may seem incoherent but their internal coherence can be made apparent once a variation criterion is identified. In a cooperative perspective, an answer can be for example: null In 2005, the average age of marriage in France was 30 for men and 28 for women.</Paragraph>
      <Paragraph position="3"> It increased by about 5.5 years between 1972 and 2005.</Paragraph>
      <Paragraph position="4"> This answer is composed of:  1. a direct answer to the question, 2. an explanation characterizing the variation mode of the numerical value.</Paragraph>
      <Paragraph position="5">  To generate this kind of answer, it is necessary (1) to integrate candidate answers in order to elaborate a direct answer (for example by solving inconsistencies), and (2) to integrate candidate answers characteristics in order to generate an explanation. Figure 1 presents the general architecture of our system which allows us to generate answers and explanations from several different numerical answers. Questions are submitted in natural language to QRISTAL1 which analyses them and selects potential answers from the Web. Then, a grammar is applied to extract information needed for the generation of an appropriate cooperative answer. This information is mainly:  - the searched numerical value (val), - the unit of measure, - the question focus, - the date and place of the information, - the restriction(s) on the question focus , - the precision of the numerical value (for example adverbs or prepositions such as in about 700, ...), - linguistic clues indicating a variation of the value  (temporal adverbs, verbs of change/movement as in the price increased to 200 euro).</Paragraph>
      <Paragraph position="6"> For the extraction of restrictions, a set of basic properties is defined (colors, form, material, etc.). Ontologies are also necessary. For example, for the question how many inhabitants are there in France?, population of overseas regions and metropolitan population are restrictions of France because they are daughters of the concept France in the ontology. On the contrary, prison population of France is not a restriction because prison is not a daughter of France. Several ontologies are available2 but the lack of available knowledge for  some domains obviously influences the quality of answers.</Paragraph>
      <Paragraph position="7"> We define the set A = fa1;:::;aNg, with ai a frame which gathers all this information for a numerical value. Figure 2 shows an extraction result.  From the frame set, the variation criteria and mode of the searched numerical value are identified: these components perform content determination. Finally, a natural language answer is generated explaining those characteristics. Each of these stages is presented in the next sections.</Paragraph>
    </Section>
  </Section>
  <Section position="7" start_page="104" end_page="105" type="metho">
    <SectionTitle>
3 Content determination for
</SectionTitle>
    <Paragraph position="0"> explanations In order to produce explanations for data variation, the system must have a data analysis component  which can infer, from extracted information, the variation phenomena, criteria and mode.</Paragraph>
    <Section position="1" start_page="105" end_page="105" type="sub_section">
      <SectionTitle>
3.1 Variation criteria
</SectionTitle>
      <Paragraph position="0"> Once we have the frames representing the different numerical values, the goal is to determine if there is a variation and to identify the variation criteria of the value. We assume that there is a variation if there is at least k different numerical values with different criteria (time, place, restriction) among the N frames (for the moment, we arbitrarily set k = N=4, but this has to be evaluated). Thus, a numerical value varies according to: 1. time if T = fai(V al); 9 aj 2 A; such as ai(V al) 6= aj(V al)</Paragraph>
      <Paragraph position="2"> 4. time and place if (1) ^ (2) 5. time and restriction if (1) ^ (3) 6. place and restriction if (2) ^ (3) 7. time, place and restriction if (1)^(2)^(3)  Numerical values can be compared only if they have the same unit of measure. If not, they have to be converted. More details about comparison rules are presented in (Moriceau, 2006).</Paragraph>
    </Section>
    <Section position="2" start_page="105" end_page="105" type="sub_section">
      <SectionTitle>
3.2 Variation mode
</SectionTitle>
      <Paragraph position="0"> In the case of numerical values varying over time, it is possible to characterize more precisely the variation. The idea is to draw a trend (increase, decrease, ...) of variaton over time so that a precise explanation can be generated. For this purpose, we draw a regression line which determines the relationship between the two extracted variables value and date.</Paragraph>
      <Paragraph position="1"> In particular, Pearson's correlation coefficient (r), related to the line slope, reflects the degree of linear relationship between two variables. It ranges from +1 to 1. For example, figure 3 shows that a positive Pearson's correlation implies a general increase of values whereas a negative Pearson's correlation implies a general decrease. On the contrary, if r is low ( 0:6 &lt; r &lt; 0:6), then we consider that the variation is random (Fisher, 1925).</Paragraph>
      <Paragraph position="2">  are there in France?</Paragraph>
    </Section>
  </Section>
  <Section position="8" start_page="105" end_page="108" type="metho">
    <SectionTitle>
4 Answer generation
</SectionTitle>
    <Paragraph position="0"> Once the searched numerical values have been extracted and characterized by their variation criteria and mode, a cooperative answer is generated in natural language. It is composed of two parts: - a direct answer if available, - an explanation of the value variation.</Paragraph>
    <Section position="1" start_page="105" end_page="106" type="sub_section">
      <SectionTitle>
4.1 Direct answer generation
</SectionTitle>
      <Paragraph position="0"> The content determination process for the direct answer generation is mainly guided by constraints which may be explicit or implicit in the question. For example, in the question how many inhabitants are there in France in 2006?, there  are explicit constraints on time and place. On the contrary, in how many inhabitants are there in France?, there is no constraint on time. Let C be the set of question constraints: C = fCt;Cp;Crg  with : - Ct: constraint on time (Ct 2 fexp time;;g), - Cp: constraint on place (Cp 2 fexp place;;g), - Cr: constraint on restrictions (Cr 2 fexp restr;  ;g).</Paragraph>
      <Paragraph position="1"> For example, in the question what is the average age of marriage in France?: Ct = ;; Cp = France and Cr = ;.</Paragraph>
      <Paragraph position="2"> When there is no explicit constraint in the question, we distinguish several cases: - if there is no explicit constraint on time in the question and if a numerical variation over time has been infered from the data set, then we assume that the user wants to have the most recent information: Ct = max(fai(date); ai 2 Ag), - if there is no explicit constraint on place in the question and if a numerical variation according to place has been infered from the data set, then we assume that the user wants to have the information for the closest place to him (the system can have this information for example via a user model), - if there is no explicit constraint on restrictions in the question and if a numerical variation according to restrictions has been infered from the data set, then we assume that the user wants to have the information for any restrictions.</Paragraph>
      <Paragraph position="3"> For example, on figure 5: Ct = 2000 (the most recent information), Cp = France and Cr = ;.</Paragraph>
      <Paragraph position="4">  Candidate frames for direct answers are those which satisfy the set of constraints C. Let AC be the set of frames which satisfy C (via subsumption): null</Paragraph>
      <Paragraph position="6"> For figure 5: AC = fa1;a2;a3;a4;a5;a6g.</Paragraph>
    </Section>
    <Section position="2" start_page="106" end_page="107" type="sub_section">
      <SectionTitle>
4.1.3 Choosing a direct answer
</SectionTitle>
      <Paragraph position="0"> A direct answer has to be generated from the set AC. We define subsets of AC which contain frames having the same restrictions: a direct answer will be generated for each relevant restriction. Let A be the subsets of frames satisfying the question constraints and having the</Paragraph>
      <Paragraph position="2"> Then, for each element in A , an answer is</Paragraph>
      <Paragraph position="4"> Each element of A may contain one or several frames, i.e. one or several numerical data.</Paragraph>
      <Paragraph position="5"> Some of these values may be aberrant (for example, How high is the Eiffel Tower? 300m, 324m, 18cm): they are filtered out via classical statistical methods (use of the standard deviation). Among the remaining frames, values may be equal or not at different degrees (rounded values, for example).</Paragraph>
      <Paragraph position="6"> Those values have to be integrated so that a synthetic answer can be generated.</Paragraph>
      <Paragraph position="7"> There are many operators used in logical approaches for fusion: conjunction, disjunction, average, etc. But, they may produce an answer which is not cooperative: a conjunction or disjunction of all candidates may mislead users; the average of candidates is an &amp;quot;artificial&amp;quot; answer since it has been computed and not extracted from Web pages.</Paragraph>
      <Paragraph position="8"> Our approach allows the system to choose a value among the set of possible values, dealing with the problem of rounded or approximative data. Candidate values are represented by an oriented graph whose arcs are weighted with the cost between the two linked values and the weight (w) of the departure value (its number of occurrences).</Paragraph>
      <Paragraph position="9"> A graph G of numerical values is defined by N the set of nodes (set of values) and Arc the set of arcs. The cost c(x;y) of arc(x;y) is:</Paragraph>
      <Paragraph position="11"> c(xi;x): with (x1;:::;xn;x) a path from x1 to x. Finally, we define a fusion operator which selects the value which is used for the direct answer. This value is the one which maximizes the difference (cost(x)) between the cost to leave this value and the cost to arrive to this value:</Paragraph>
      <Paragraph position="13"> with: cost leave(x) =Pi c(x;xi) and, cost arrive(x) =Pi c(xi;x).</Paragraph>
      <Paragraph position="14"> Let us consider an example. The following values are candidate for the direct answer to the question How high is the Mont-Blanc?: 4800, 4807 (2 occurrences), 4808 (2 occurrences), 4808.75, 4810 (8 occurrences) and 4813. Figure 6 shows the graph of values: in this example, the value which maximizes the costs is 4810.</Paragraph>
      <Paragraph position="15"> From the selected value, the system generates a direct answer in natural language in the form of Focus Verb (Precision) Value. For example, the generated answer for How high is the Mont-Blanc? is The Mont-Blanc is about 4810 meters high. Here the preposition about indicates to the user that the given value is an approximation.</Paragraph>
      <Paragraph position="16"> For the question what is the average age of marriage in France?, a direct answer has to be generated for each restriction. For the restriction men (AC2), there are 3 candidate values: 29.8, 30 and 30.6, the value which minimizes the costs being 30. For the restriction women (AC1), there are also 3 candidate values: 27.7, 28 and 28.5, the value which minimizes the costs being 28. After aggregation process, the generated direct answer is: In 2000, the average age of marriage in France was about 30 years for men and 28 years for women.</Paragraph>
    </Section>
    <Section position="3" start_page="107" end_page="108" type="sub_section">
      <SectionTitle>
4.2 Explanation generation
</SectionTitle>
      <Paragraph position="0"> The generation of the cooperative part of the answer is complex because it requires lexical knowledge. This part of the answer has to explain to the user variation phenomena of search values: when a variation of values is identified and characterised, an explanation is generated in the form of X varies according to Criteria. In the case of variation according to restrictions or properties of the focus, a generalizer is generated. For example, the average age of marriage varies for men and women: the explanation is in the form the average age of marriage varies according to sex. The generalizer is the mother concept in the ontology or a property of the mother concept (Benamara, 2004).</Paragraph>
      <Paragraph position="1"> For numerical value varying over time, if the variation mode (increase or decrease) is identified, a more precise explanation is generated: X increased/decreased between... and... instead of X varies over time.</Paragraph>
      <Paragraph position="2"> Here, verbs are used to express precisely numerical variations. The lexicalisation process needs deep lexical descriptions. We use for that purpose a classification of French verbs (Saint-Dizier, 1999) based on the main classes defined by Word-Net. The classes we are interested in for our task are mainly those of verbs of state (have, be, weight, etc.), verbs of change (increase, decrease, etc.) and verbs of movement (climb, move forward/backward, etc.) used metaphorically (Moriceau et al, 2003). From these classes, we selected a set of about 100 verbs which can be applied to numerical values.</Paragraph>
      <Paragraph position="3"> From these classes, we characterized sub-classes of growth, decrease, etc., so that the lexicalisation task is constrained by the type of verbs which has to be used according to the variation mode.</Paragraph>
      <Paragraph position="4"> A deep semantics of verbs is necessary to generate an answer which takes into account the characteristics of numerical variation as well as possible: for example, the variation mode but also the speed and range of the variation. Thus, for each sub-class of verbs and its associated variation mode, we need a refined description of ontological domains and selectional restrictions so that  an appropriate verb lexicalisation can be chosen: which verb can be applied to prices, to age, etc.? (Moriceau et al, 2003). We propose to use proportional series representing verb sub-classes according to the speed and amplitude of variation. For example, the use of climb (resp. drop) indicates a faster growth (resp. decrease) than go up (resp.</Paragraph>
      <Paragraph position="5"> go down): the verb climb is prefered for the generation of Gas prices climb 20.3% in october 2005 whereas go up is prefered in Gas prices went up 7.2% in september 2005.</Paragraph>
      <Paragraph position="6"> Verbs can possibly be associated with a preposition that refines the information (The average age of marriage increased by about 5.5 years between 1972 and 2005).</Paragraph>
    </Section>
    <Section position="4" start_page="108" end_page="108" type="sub_section">
      <SectionTitle>
4.3 Answer justification
</SectionTitle>
      <Paragraph position="0"> Our system generates a cooperative answer composed of a direct answer to the question and an explanation for the possible variation of the searched numerical value. But the answer may not be sure because of a too high/low number of candidate values to the direct answer. In this case, it may be useful to add some additional information for the user in order to justify or complete the generated answer.</Paragraph>
      <Paragraph position="1"> We propose to add a know-how component to our system, which provides the user with one or two relevant Web page extracts besides the generated answer whenever it is necessary. These extracts must contain information about the searched numerical values, and for example some explanations of the causes of numerical variation. Some linguistic clues can be used to select page extracts: number of numerical values concerning the question focus, causal marks (because of, due to, ...), etc. Figure 7 shows an output example of our system. null</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>