File Information
File: 05-lr/acl_arc_1_sum/cleansed_text/xml_by_section/metho/91/h91-1070_metho.xml
Size: 19,205 bytes
Last Modified: 2025-10-06 14:12:42
<?xml version="1.0" standalone="yes"?> <Paper uid="H91-1070"> <Title>Interactive Problem Solving and Dialogue in the ATIS Domain 1</Title> <Section position="3" start_page="0" end_page="355" type="metho"> <SectionTitle> MODELLING METHOD OLOGY </SectionTitle> <Paragraph position="0"> The back-end component of the MIT ATm system has been completely redesigned since last June \[5\]. The main system is described in detail in \[4\], and will only be briefly parse tree for the sentence, &quot;Does flight twenty two serve dinner?&quot; mentioned here. The parser delivers a semantic frame to the back-end, which processes it to produce a verbal response and a database table for display. Processing includes a step to incorporate previous elements from thehistory that may still hold, even though they were not explicitly mentioned in the immediate sentence. An example frame is given in Figure 1 for the sentence &quot;Does flight twenty two serve dinner?&quot; The system can be operated in both a non-booking and a booking mode. 2 In the former, when a user tries to make a reservation, he/she is simply informed that such a capability does not yet exist. In the latter, the system launches a reservations plan upon user request, which includes a number of subgoals initiated by either the system or the user.</Paragraph> <Paragraph position="1"> Once a user initiates a booking, a complex series of events transpires, in which the computer is actively interpreting the state of the ticket and initiating both explicit requests to the user and calls to the database to provide relevant additional information. It also displays a facsimile of the ticket (see Figure 3 below), and slots get filled in as they become specified. The computer can carry the user all the way through a round trip ticket, being sure to get unique flight/fare/date specifications for both legs, and making sure that the dates are not violated by fare restrictions. It also warns the user about the date limits for the return flight when they try to book a restricted fare.</Paragraph> <Paragraph position="2"> In our current system, we keep at most two distinct flight events in the history table. One of these refers roughly to the most recently mentioned set of flights requested by the user, appearing as a new object for reference. The other flight event in the history records the most recently referenced unique flight or itemized flight set, typically introduced when the user specifically asks for more information about a flight that was previously presented in a table. 3 Flight events are not inherited wholesale except through specific ~We define booking as the process of acquiring all relevant information for an itinerary, including the fare. At the moment, no further action, such as seat assignment and purchase/issue of the ticket, is performed, although it could presumably be simulated at a later time. 3This is a more limited approach than a general stack of available anaphoric reference such as &quot;it&quot; or &quot;these flights.&quot; Instead, individual modifiers are inherited unless new modifiers override their inheritance. History elements are stored in the standard frame format, and inheritance of a modifier usually amounts to simply inserting it into the appropriate frame of the new sentence.</Paragraph> <Paragraph position="3"> A flight that is incompletely specified in a new sentence inherits modifiers that are consistent with its current state.</Paragraph> <Paragraph position="4"> Defining consistency is tricky and requires knowledge of how information is structured in the domain. Within the ATIS domain, the explicit mention of a flight number is taken to mark a change of focus, and therefore blocks almost all inheritance modifiers except source and destination. Similarly, the modifier &quot;cheapest&quot; would block an inheritance of a specific flight number, since it implies taking a subset of a previously mentioned set. Whenever both a new source and a new destination are present, all inheritance is blocked, unless the new sentence was a clarifier, such as &quot;How about from Boston to Denver?&quot; Of course, a modifier always blocks inheritance of an entry under the same key in the history table. Exactly which modifiers should block which others was determined empirically from subject data through the data collection episodes.</Paragraph> <Paragraph position="5"> The history table contains not only the frames associated with previously mentioned noun phrases and their modifiers, but also the previously displayed table, the previous state of the ticket under development, and previously booked tickets or first legs of a round trip ticket. The system frequently consults the tickets, as well as other elements from the history, to decide what directed questions to ask the user.</Paragraph> <Paragraph position="6"> Occasionally, the history elements have to be reinterpreted after being inherited. This is particularly true for &quot;return&quot; flights, which can be mentioned in a number of different ways: using an adjective or a verb phrase modifier, with or without explicit mention of a source, destination, or date, and with or without a mention of a forward leg in the same sentence.</Paragraph> <Paragraph position="7"> It turned out to be quite difficult to make all conditions work out, inheriting source and destination when appropriate, and reversing them only if they came fl'om a history flight that was not also marked as a return flight. Return flights also inherit fare class, fare restrictions, and airline. In addition, a restricted weekday fare is generalized to include a compatible weekend fare, and some restrictions require a minimum/maximum stay restricting the return date. Finally, users often mention the return date early in the dialogue, in which case the system stores it and recalls it later, when the topic turns to the return leg.</Paragraph> <Paragraph position="8"> Figure 2 gives a block diagram of the control flow for managing discourse and dialogue. As shown in the figure, both the user and the computer may issue questions to the back-end component. These questions are processed the same discourse references \[I\]. However it has been sufficient for the nTIS domain to date.</Paragraph> <Paragraph position="9"> way, updating both the discourse and dialogue components accordingly. For instance, when the user has booked a particular flight but has said nothing about fares, the system can simply issue the request &quot;Show fares&quot; to the back-end. The discourse history will incorporate automatically the relevant flight information. If a user query is ambiguous, the system defers calling the database until it has queried the user for resolution of the ambiguity. After the computer has answered the user's question, it assesses the dialogue state, which is maintained as a stack. When the state stack is popped, the system may update the information contained in the ticket.</Paragraph> <Paragraph position="10"> The computer may decide at this point to take the initiative, anticipating the user's needs.</Paragraph> <Paragraph position="11"> Consider the example in which the user says, &quot;Book the cheapest flight.&quot; The system does not immediately know whether it should find the cheapest one-way fare or the cheapest round-trip fare. It also must remember, however, that a booking has been requested. The system pushes <booking> onto the dialogue state, followed by <resolve flight cycle>.</Paragraph> <Paragraph position="12"> This is similar to the stack-based approach described in \[2\].</Paragraph> <Paragraph position="13"> The database query function examines the top of the dialogue state and finds that more information is needed before a table can be displayed, so it does nothing. Control now passes to the computer, which asks the directed question, &quot;One way or round trip?&quot; and pops the top of the dialogue stack. After the question has been answered by the user, the user's answer is incorporated into the flight-event object, and a table is displayed. Now it consults the dialogue state once again and finds <booking>, so it calls up the booking routine to fill in all the ticket information and make the next decision about what to ask.</Paragraph> <Paragraph position="14"> Before deciding to query the user about the one-way/roundtrip ambiguity, the computer tries hard to infer the answer from history information. Of course, if this had been specified in a previous sentence, then it would be available as a flight-modifier frame in the history. It is also possible that the table previously displayed contained only round-trip (or only one-way) fares, in which case it could decide based on the table. Finally, if the user had previously specified a return date (as in the example below), the computer would assume a round-trip fare was wanted. Only when all these conditions fail does it resort to asking the user.</Paragraph> </Section> <Section position="4" start_page="355" end_page="357" type="metho"> <SectionTitle> AN EXAMPLE </SectionTitle> <Paragraph position="0"> The easiest way to explain some of the discourse/dialogue aspects of the system is through an example of a simulated dialogue, as-shown in Table 14 . The dialogue involves making a round trip reservation fl-om Boston to San Francisco. The subject's first sentence indicates only a destination. Rather than displaying all flights to San Francisco from anywhere, the system asks instead a directed question for clarification of the source location. The subject responds with only the word &quot;Boston,&quot; but the system infers &quot;from Boston&quot; based on the dialogue state. The system then immediately asks for a date, since this will dictate whether the flight is available on that day and also which fares apply. The next sentence from the subject, &quot;I'll be leaving Boston next Sunday and returning the following Tuesday,&quot; provides two dates, the first relative to today's date and the second relative to the first one. The system updates the reference date &quot;on the fly&quot; so that the appropriate date is available for reference when the phrase &quot;the following Tuesday&quot; is processed. The system now answers by focusing on the forward leg, holding the return date for later reference. It also provides verbal feedback verifying its understanding of the flight conditions: &quot;These are the flights from Boston to San Francisco on Sunday, January 6,&quot; where it has interpreted &quot;next Sunday&quot; as &quot;Sunday of next week.&quot; Finally, it fills in the source, destination, and date on the ticket form being displayed continuously to the subject.</Paragraph> <Paragraph position="1"> The subject now asks a question referring to a specific entry in the list, &quot;the third one.&quot; The system extracts airline, flight number, source and destination from the table entry, and constructs the appropriate semantic frame, inheriting the date from the history. Again it confirms understanding with a verbal response.</Paragraph> <Paragraph position="2"> Question 5 from the subject is a complicated one, containing two superlative adjectives. The system knows to first extract all non-stop flights that serve dinner 5, then to pick the earliest among them; and finally to find the cheapest fare for that flight. If these modifiers were processed in a different order, it would not get the right answer. Usually the system would ask for a one-way/round-trip disambiguation at this point, but since the subject has already specified a return date, the system assumes they want a round-trip ticket. The cheapest fare is an unrestricted one because the restrictions are failing on advance-purchase requirements, given the date.</Paragraph> <Paragraph position="3"> In Question 6, the subject says, &quot;Book it,&quot; where &quot;it&quot; appears after Question 6 in the simulated dialogue from Table 1.</Paragraph> <Paragraph position="4"> could be the flight or the fare. The system assumes &quot;it&quot; means the entire noun phrase in the system's answer (&quot;the cheapest round-trip fare for the earliest non-stop flights&quot;), i.e., the fare along with the flight restrictions implied by the for-phrase. The system then fills in the appropriate slots in the displayed ticket, including airline, flight number, departure and arrival times, far e category, and dollar amount.</Paragraph> <Paragraph position="5"> When the system displays the table this time, it says, &quot;I'll show it to you again,&quot; rather than the usual &quot;This is the <fare with appropriate description>.&quot; This represents an attempt to reduce the verbose nature of the computer responses, done only on the condition that the sentence about to be spoken is identical to the one the computer just said. Figure 3 shows a reproduction of the ticket as it appears after Question 6 has been processed.</Paragraph> <Paragraph position="6"> By examining the ticket, the system determines that there are both a unique flight and a unique fare available for booking. Had there only been flight information specified, the system would have taken the initiative to display the fare Options for that flight on that date, and to ask the subject to &quot;narrow down the fare to a single choice.&quot; The system now says, &quot;I'll book United flight 93 from Boston to San Francisco for you,&quot; thus renaming &quot;the earliest non-stop flight serving dinner.&quot; At this point, the system reminds the subject that there is a return leg, and also that the date, &quot;Tuesday January 8&quot; had been previously specified. Even if the subject had not mentioned a return date, the system would still ask whether it could help with the return flight. Furthermore, when the subject selects a restricted fare, the system warns them about the earliest and latest dates they are allowed to return on, and rejects a return date whenever it is outside this range.</Paragraph> <Paragraph position="7"> The subject has two general options for a &quot;yes&quot; answer to the question, &quot;Can I help you with the return flight on Tuesday January 87&quot; One is a simple, &quot;Yes please,&quot; in which case the system Constructs the appropriate restrictions based on the ticket slots. The other is a direct statement explicitly asking for return flights, such as &quot;Show me the return flights,&quot; or &quot;I'd like to see flights returning on January ninth,&quot; in which case it inherits appropriate information from the semantic frame in the history table, reversing source and destination.</Paragraph> <Paragraph position="8"> The system now shows the subject two United flights from San Francisco to Boston, and asks the subject to select one.</Paragraph> <Paragraph position="9"> The subject selects flight 92, and the system is now ready to form a complete booking consisting of two flights tied to a single fare, adding it to a list of previous bookings. The system finally asks the subject, &quot;Can I help you with something else?&quot; and the subject concludes the dialogue.</Paragraph> </Section> <Section position="5" start_page="357" end_page="357" type="metho"> <SectionTitle> COLLECTING DIALOGUE DATA </SectionTitle> <Paragraph position="0"> We have collected several thousand sentences from subjects using our system \[3\], but only about ten of the subjects were allowed to use the system in booking mode. Even for these ten, we only asked them to do one booking scenario, in addition to several non-booking scenarios. In part, this was done because data collection at TI is done in non-booking mode, and we wanted our data to be better matched to the likely TI test data. In addition, we were not confident that our booking dialogue was sufficiently robust to be ready for data collection until the last month or so. We are encouraged, however, by preliminary results of the booking scenarios. While a subject almost never gets through a booking &quot;without a hitch,&quot; we do find that subjects are able to specify flights to be booked and successfully complete scenarios.</Paragraph> <Paragraph position="1"> With each data collection episode, we gain new insights on faulty assumptions in the system. A close coupling between data collection and system development should ultimately yield a robust dialogue model, which could possibly be viewed as an &quot;artist's conception&quot; of a useful system.</Paragraph> <Paragraph position="2"> In the appendix can be found an example of an actual dialogue between one of our subjects and the computer. Twice during the dialogue the computer made faulty assumptions, but the subject was able to recover from the error and ultimately achieve all the goals of the scenario. Obviously, we are improving the system so that, next time around, these errors will not reccur.</Paragraph> </Section> <Section position="6" start_page="357" end_page="357" type="metho"> <SectionTitle> HELPING THE RECOGNIZER </SectionTitle> <Paragraph position="0"> There are potentially many ways to use discourse and dialogue to make the speech recognition task more successful.</Paragraph> <Paragraph position="1"> A simple step that we are taking is to restrict flight numbers to be only those that have previously been displayed in a table. Hypotheses with unlicensed flight numbers would be pruned away. This can be effective, since numbers are relatively difficult to recognize correctly.</Paragraph> <Paragraph position="2"> Another feature that we have implemented has to do with ease of recovery from a recognition error. A system which remembers the history can become quite confused if it remembers false information from sentences that were incorrectly recognized. We have therefore implemented a &quot;scratch that&quot; or &quot;erase that&quot; command which allows the system to completely &quot;forget&quot; all information that was newly introduced in the most recent sentence. This includes erasing entries from the ticket display if the sentence requested a booking. This command is distinct from a &quot;clear history&quot; command, which has a more global effect of erasing all records in the history.</Paragraph> <Paragraph position="3"> Finally, we hope to be able to use the dialogue state to dynamically modify probabilities on arcs in the grammar. Once we have a fully integrated recognizer, with parse probabilities incorporated into the scores of partial theories, perplexity can be reduced by rewarding paths that are supported by the dialogue state. For example, we could introduce a bonus on the <date> node; whenever the system asks the question, &quot;What date will you be travelling on?&quot; We hope to be able to explore some of tlhese ideas in the near future.</Paragraph> </Section> class="xml-element"></Paper>