<?xml version="1.0" standalone="yes"?>
<Paper uid="A83-1031">
  <Title>INTERACTIVE NATURAL LANGUAGE PROBLEM SOLVING: A PRAGMATIC APPROACH</Title>
  <Section position="1" start_page="0" end_page="0" type="abstr">
    <SectionTitle>
ABSTRACT
I NTRODUCT ION
</SectionTitle>
    <Paragraph position="0"> A class of natural language processors is described which allow a user to display objects of interest on a computer terminal and manipulate them via typed or spoken English sentences.</Paragraph>
    <Paragraph position="1"> This paper concerns itself with the implementation of the voice input facility using an automatic speech recognizer, and the touch input facility using a touch sensitive screen. To overcome the high error rates of the speech recognizer under conditions of actual problem solving in natural language, error correction software has been designed and is described here. Also described are problems involving the resolution of voice input with touch input, and the identification of the intended referents of touch input.</Paragraph>
    <Paragraph position="2"> To measur~ system performance we have considered two classes of factors: the various conditions of testing, and the level and quality of training of the system user. In the paper a sequence of five different testing situations is observed, each one resulting in a lowering of system performance by several percentage points below the previous one. A training procedure for potential users is described. and an experiment is discussed which utilizes the training procedure to enable users to solve actual non-trivial problems using natural language voice communication. null A class of natural language processors is under development which allow a user to display objects of interest on a computer terminal and manipulate them via typed or spoken English imperative sentences. Such a processor is designed to respond within one to four seconds by executing the input command and updating the displayed world for user verification. If an undesired action is observed, a &amp;quot;backup&amp;quot; command makes it possible to undo any action and return the system to a previous state. The domains of interest include matrix computation, where one can display tables of data and manipulate them: office automation, where one can work with texts, files, calendars, or messages: and machine control, where one might wish to command a robot or other equipment via natural language input.</Paragraph>
    <Paragraph position="3"> The first such system (Biermann and Ballard \[6\]), called NLC, provides a matrix computation facility and allows users to display matrices, enter data, and manipulate the entries, rows, and columns.</Paragraph>
    <Paragraph position="4"> It became operative in Ig79 and includes a variety of special purpose features 1 ~his Work was supported by National Science Foundation Grants MCS 7904120 and MCS 8113491, by the IBM Corporation under GSD agreement no. 260880, and by the Universite de Paris-Sud, Laboratoire de Recherche en Informatique during the summer of Ig82.</Paragraph>
    <Paragraph position="5">  including arbitrarily deep nesting of noun groups, extensive conjunction processing, user defined imperative verbs, and looping and branching features. More recently, a domain independent abstraction of the NLC system has been constructed and now is being specialized to handle a text processlng task. In this system, text can be displayed and modified or formatted with natural language commands.</Paragraph>
    <Paragraph position="6"> Current work emphasizes the addition of voice input, voice output, and a touch sensitive display screen. Speech recognition is being done on an experimental basis with the Nippon Electric DP-200 Connected Speech Recognizer in both discrete and connected speech modes, and with the</Paragraph>
    <Section position="1" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
Votan Corporation V-SO00 Development Sys-
</SectionTitle>
      <Paragraph position="0"> tem. The touch sensitive screen being used is a Carroll touch panel mounted on a 19-inch color monitor. Voice response is also provided by the Votan V-5000 which assembles and vocalizes digitally recorded human voice messages. The work has progressed to the point where OUr natural language matrix computer NLC is operative under voice control using the DP-200 and the text processing system is beginning to function using the V-5000 speech recognizer. The touch panel interface and voice response systems are still in the design phase.</Paragraph>
      <Paragraph position="1"> The goal of the project is to make possible voice and touch interactions of the following kind: Retrieve file Budget83.</Paragraph>
      <Paragraph position="2"> Find the largest number in this column and zero it. (with touch input) Add this column putting the result here. (with two touch inputs) Send this file to Jones and file it as Budget83. (touch input) ~at is, imperative sentences are to be processed that operate on domain objects to produce modifications to the existing objects or their relationship to each other. The objects are, for example, rows, columns, numbers, entries, labels, etc. in the matrix domain or sections, paragraphs, sentences, margins, pages, etc. in the text processing domain. The execution of each command is accompanied by an update of the displayed data with highlighting to indicate changes. Prompts and error messages will be given by voice response, gystem design is aimed at allowing fast interactive control of the objects on the screen while the user maintains uninterrupted eye contact with th~ events as they happen.</Paragraph>
      <Paragraph position="3"> A continuous program of human factors testing has been maintained by the project in order to build a realistic view of potential users and to measure Progress in achieving usability. For example, in a test of the matrix computation system with typed input, twenty-three subjects solved problems similar to those that might be assigned in a first course in programming (Biermann, Ballard, and Sigmon \[7\]). In this test, the NLC system correctly processed 81 percent of the sentences and users were quite satisfied with its general performance. Other tests of the system are described in Fink \[14\] and Geist et el. \[IS\]. In another test (Fineman \[13\]), a simulator for a voice driven office automation system was used to obtain data on user behaviors when problem solving is with discrete and slow connetted speech. It was found that users quickly adapted their speech to the required discipline of slow, methodical, and simple sentences which can be recognized by machine. Since the data obtained in any system test is heavily dependent on the amount and kind of training given to subjects, it is necessary to have a standardlzed training procedure. In the current work, a voice tutorial has been developed for training users to use a voice interactive system (Deas \[Ii\]).</Paragraph>
      <Paragraph position="4"> This paper reports on the current status of these projects with emphasis on system design, speech input facilities and their performance, the touch input system and human factors considerations.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>