<?xml version="1.0" standalone="yes"?>
<Paper uid="A97-1002">
  <Title>Natural Language in Four Spatial Interfaces</Title>
  <Section position="5" start_page="8" end_page="8" type="metho">
    <SectionTitle>
3 Application Projects
</SectionTitle>
    <Paragraph position="0"/>
    <Section position="1" start_page="8" end_page="8" type="sub_section">
      <SectionTitle>
3.1 Eucalyptus
</SectionTitle>
      <Paragraph position="0"> NAUTILUS was first used in the Eucalyptus (Wauchope, 1994) spoken language interface to the</Paragraph>
    </Section>
  </Section>
  <Section position="6" start_page="8" end_page="10" type="metho">
    <SectionTitle>
KOALAS Airborne Early Warning C2 simulation
</SectionTitle>
    <Paragraph position="0"> (Barrett and Aldrich, 1990). The original KOALAS interface consisisted of a mouse-sensitive simulated radar screen with a conventional graphical user interface composed of command pushbuttons, dialog boxes and scrolling display windows. Our objective in Eucalyptus was to make the same command and data access functionality available via natural language, integrated as much as possible with the graphical interface to allow multimodal interactions.</Paragraph>
    <Paragraph position="1"> For example, a NL command to the system might result in the display of the same dialog box used in the corresponding GUI command, but with the dialog's data fields fully or partially filled from the NL input;  the user can then fill in any remaining empty fields and issue final acceptance either graphically or verbally. To that end, the command-oriented Interface Functions in Eucalyptus consist largely of calls to the base functions underlying the KOALAS GUI.</Paragraph>
    <Paragraph position="2"> Eucalyptus also includes deictic reference, allowing the user to click on one or more radar blips or screen locations while speaking verbal references like this fighter or these CAP stations. When a mouse click occurs, NAUTILUS asks the application for the identities of all the objects located at or near the mouse event, and then takes the subset of those objects that match the semantics of the verbal phrase (which can be determined from predicate context as well: for example the word here in Have fighter I refuel here necessarily refers to a tanker aircraft). To avoid the problems of time-correlating speech with graphical input and distinguishing anaphora from deixis, we reserve the words this, these and here for deictic reference and that, those and there for anaphoric reference, and allow no more than one plural deictic reference per utterance.</Paragraph>
    <Paragraph position="3"> Database query is used both in answering explicit interrogatives (Which fighters aren't holding CAP station?) as well as dereferencing qualified NPs (moving aircraft). The system can also interpret NP sentence fragments as followup commands or queries by substituting the NP into the semantically relevant slot of the prior utterance's logical form.</Paragraph>
    <Paragraph position="4"> As originally designed, FOCAL expected a closed-world model of all domain objects to be available at startup time. This had to be modified somewhat in Eucalyptus since the KOALAS world includes hypothetical objects (suspected threat aircraft) which the user and system can create and destroy at will.</Paragraph>
    <Paragraph position="5"> References to hypothetical entities are resolved by having FOCAL dynamically consult the application database for the current object population at the time the phrase was uttered.</Paragraph>
    <Paragraph position="6"> The core syntactic grammar of about 150 context-free rules and 50 restriction rules developed for Eucalyptus has been re-used in all the other NAUTILUS projects. Each one has augmented it with a few dozen additional rules for handling application-specific constructs like station two sector one (Eucalyptus), latitude forty degrees north longitude ninety five degrees west (InterVR), the town of Leipzig (InterLACE), and thirty degrees left (InterROB).</Paragraph>
    <Paragraph position="7"> The Eucalyptus lexicon totals about 425 words, many of them unused morphological variants generated automatically by the PROTEUS lexical macros; by comparison, the vocabulary for the speech recognition front end is only 260 words. Total input coverage is on the order of 100 million utterances, deliberately high to test the speech system's ability to detect a wide variety of noun phrase determiners. Unlike the NAUTILUS grammar, the speech grammar excludes iteration and recursion (such as compounding) to maintain a reasonable level of recognition accuracy. An experimental addition of relative clauses to the speech grammar was &amp;quot;productive&amp;quot; only in the linguistic sense, since the resulting exponential increase in grammar size caused recognition rates to drop to unacceptable levels. null</Paragraph>
    <Section position="1" start_page="9" end_page="9" type="sub_section">
      <SectionTitle>
3.2 InterVR
</SectionTitle>
      <Paragraph position="0"> We next used NAUTILUS in InterVR (Everett et al., 1994), a spoken language controller for an immersive 3D or &amp;quot;Virtual Reality&amp;quot; tactical combat simulation viewer. Here the emphasis was on the utility of speech I/O in an eyes- and hands-busy virtual environment. Like Eucalyptus, InterVR supports commands, queries, complex reference and anaphora, and NP followups. A non-immersive desktop version of the viewer allowed mouse selection of a platform and thus singular deictic reference (this helicopter), but the immersive display version did not include a dataglove or other pointing device. We did not have time or resources to tackle the problem of resolving referents based on visual context (for example having that helicopter refer to the one nearest the center of the user's field of view), but we are currently investigating the interaction of vision and language in the InterROB project, to be discussed shortly.</Paragraph>
      <Paragraph position="1"> The InterVR speech component has a vocabulary comparable in size to that of Eucalyptus but a more constrained input range (about 1 million utterances) mainly due to a less liberal variety of NP determiners. The IPC code developed for Eucalyptus ported immediately to the new application, and NAUTILUS's modular architecture allowed speech modeling, NLP knowledge base development and IF coding to be pursued independently by different team members with a minimum of coordination.</Paragraph>
    </Section>
    <Section position="2" start_page="9" end_page="10" type="sub_section">
      <SectionTitle>
3.3 InterLACE
</SectionTitle>
      <Paragraph position="0"> InterLACE (Wauchope, 1996) is an integrated natural language interface and graphical map display for the Air Force's LACE land/air combat simulation system (Anken, 1989). LACE includes a large object-oriented cartographic database of most of central Germany, containing a total of over 12,000 objects such as towns, lakes, rivers, and railroads.</Paragraph>
      <Paragraph position="1"> Since the application is written in Common LISP and so can run in the same process environment as NAUTILUS, we dispensed with independently modeling FOCAL entities for the domain and just let the  LACE database objects serve as the extensions of referring expressions. To avoid having to enter hundreds of foreign proper names into the PROTEUS lexicon, we modified the PROTEUS lexical tagger to assume that any input word might be a proper name (applying that assumption only to non-English words failed the first time we encountered the river Main and the Czech towns of Most and As). Deictic reference operates similarly to Eucalyptus: a mouse click can select a number of overlapping map objects at once, to be resolved by an accompanying verbal reference; for example What's the population here? would resolve to a town object whereas Does this cross the Elbe ? might resolve to a road.</Paragraph>
      <Paragraph position="2"> The InterLACE domain necessitated extending FUNTRAN to generate proper logical forms for spatial comparatives and superlatives (Is Wurzen closer than Grimma?, the closest lake to Eilenburg) and implicit and explicit reflexives (Do these roads cross \[each other\]?), and also introduced direct address (Tank 1, head north) and iterative arguments to navigation commands (Head north on road E2 for 2 km to Wurzen). An experimental study of NL inputs from novice InterLACE users showed that of 822 inputs, 14 contained typos or misspellings and 30 contained ungrammaticalities, for an illformedness rate of 5%. Of the remaining 778 utterances NAUTILUS failed to understand 23 (3%) due to incomplete coverage. null Since the PE200's phonetic rules are for American English and (unlike PROTEUS) the module cannot be tricked into recognizing unknown inputs as possible proper names, a complete speech input component for InterLACE was impractical. For demo purposes we opted to implement a 160-word speech interface containing just fifty German proper names, few enough that the recognizer doesn't have too much trouble distinguishing them, for an input coverage of about 10 million utterances. Similarly for speech output we provided the same vocabulary to DECtalk along with phonetic transcriptions to produce acceptable German pronunciations.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>