<?xml version="1.0" standalone="yes"?>
<Paper uid="P87-1006">
  <Title>GRAMl~IATICAL AND UNGRAMMATICAL STRUCTURES IN USER-ADVISER DIALOGUES1 EVIDENCE FOR SUFFICIENCY OF RESTRICTED LANGUAGES IN NATURAL LANGUAGE INTERFACES TO ADVISORY SYSTEMS. Raymonde Gulndon Aficeoelectroni~ and Computer Technology Corporation</Title>
  <Section position="4" start_page="0" end_page="41" type="metho">
    <SectionTitle>
A STUDY OF USER-ADVISER DIALOGUES
IN A WIT.ARDoOF-OZ SETTING
METIIOD ~ PROCEDURE
</SectionTitle>
    <Paragraph position="0"> Thirty-two graduate students with basic statistical knowledge were asked to solve up to eleven simple statistics problems. Participants had to use an unfamillar statistical package to solve the problems. The upper window of the participants' screen was used to perform operations with the statistical package and the lower window was used to type utterances to the adviser. The participants were instructed to ask help in English from what they believed was a computerized adviser by typing in the help window. Tile participants' and adviser's utterances were sent to each other's monitor and the utterances were recorded and time-stamped automatically to files.</Paragraph>
    <Paragraph position="1"> INow at Automated Language Processing Systems, Provo, Utah</Paragraph>
  </Section>
  <Section position="5" start_page="41" end_page="41" type="metho">
    <SectionTitle>
RESULTS AND COMPARISON TO OTIIER
STUDIES
</SectionTitle>
    <Paragraph position="0"> We are reporting only a small subset of our results, those to be compared to the results of Thompson (1980) and of Chafe (~.982). The comparison is to identify the grammatical and ungrammatical specializations specific to users' language with advisory systems and to help determine what features of user-advising situations might encourage or cause such specializations of structures. Chal'e (1982) investigated Informal Spoken language (i.e., dinner table conversations) and formal written language (i.e., academic papers). Thompson (1980), in her second study, compared three types of dialogues, Spoken Face-to-Face, Typed Human-Human (terminal-to-terminal) with both conversants knowing their counterpart was human, and Human-Computer using the REL natural language front-end. The task was information retrieval.</Paragraph>
    <Paragraph position="1"> The data table report two sets of data, the percentage of utterances with a particular form (e.g., one or more Fragments, one or more phatics) to compare to Thompson's results and the corresponding number of occurrences of this form per 1000 words to compare to Chafe's results. When numbers are omitted from the tables, the corresponding data were not collected by Thompson or Chafe. Note that the reported data are only about users' utterances, and not the adviser's utterances. We will use typed user-adviser dialogues and Wizard.of-Oz condition to refer to the data of our study.</Paragraph>
    <Section position="1" start_page="41" end_page="41" type="sub_section">
      <SectionTitle>
Completeness and Formality of Users'
Utterances
</SectionTitle>
      <Paragraph position="0"> As can be seen in Table I, for completeness (i.e., fragments) and formality (i.e., phatics and and-connectors) users' utterances with advisory systems are more like Human-Computer dialogues and Formal Written language than Spoken Face-to-Face or Typed Human-Human dialogues.</Paragraph>
      <Paragraph position="2"> Users avoided casual forms of language since they produced only 24% of fragmentary utterances, as opposed to 74% in Typed Human-Human dialogues, but similar to 19% in the Human-Computer condition. Similarly, we found 2% of utterances with phatics, as opposed to 59% in the Typed Human-Human dialogues, but similar to 4% in the Human-Computer dialogues. Likewise, Chafe \[1980) found no phatics in Formal Written discourse, but .,bout 23 per 1000 words in informal speech. There is a similar finding for andconnectors. null Users in the typed user-adviser dialogues seem to expect the interface to be unable to handle fragmentary input such ~s found in Informal Spoken language and planned or edlted their language to be as complete and formal as in the Human-Computer dialogues, and more complete and formal than the language in Typed Human-Human dialogues.</Paragraph>
      <Paragraph position="3"> This is the case even though the Wizard in our study hardly ever rejected or misunderstood any users' utterance, no matter how fragmentary or ungrammatical it was. However, when conversants know that their counterpart is another human, their language contains a large percentage of fragments and phatics, even when typed. So it appears that a priori beliefs about the nature and abilities of the adviser (i.e., this is not a human) can determine the characteristics of the language produced by the user, even when task and linguistic performances by the adviser were not negatively afletted by fragmentary language from the user.</Paragraph>
    </Section>
    <Section position="2" start_page="41" end_page="41" type="sub_section">
      <SectionTitle>
Ungrammatlealltles
</SectionTitle>
      <Paragraph position="0"> Even though users seemed to attempt to edit or plan their utterances to be more complete and formal, 31deg~ of the utterances contained one or more ungrammaticalities (excluding spelling and punctuation mistakes, if included about 50~ of utterances were ungrammatical). The most frequent ungrammaticalities were Fragments (13% of utterances with part(s) of the utterance being one or more fragments), missing constituents (14~ of utterances with one or more determiners missing), and lack of agreement between constituents (5% of utterances). While users seemed to plan or to edit their language to be as complete and formal as in the Human-Computer dialogues, certain types of ungrammaticaiities were produced. Two possible interpretations of this finding are: I) Certain types of ungrammaticalities do not seem to be easily under the conversant's control and edited or planned to be avoided during the dialogue; 2) They correspond to a telegraphic language assumed to be understood by the interface.</Paragraph>
      <Paragraph position="1"> It would be interesting to find whether really two types of ungrammaticaiities exist, some that can be avoided under some planning and others that cannot be so easily avoided.</Paragraph>
      <Paragraph position="2"> However, it is unclear whether the purposeful avoidance of some ungrammaticaiities by users can be capitalized upon to reduce the need For sophisticated robust parsing us we do not know the cost from the users of avoiding certain types of ungrammaticalities. On the other hand, knowing the nature and frequency of the actual ungrammaticaiities produced by users, as they are provided by this study, Facilitates realizing robust parsing.</Paragraph>
    </Section>
  </Section>
  <Section position="6" start_page="41" end_page="42" type="metho">
    <SectionTitle>
General Syntactic Features
</SectionTitle>
    <Paragraph position="0"> As can be seen in Table 2, users' utterances in typed user-adviser dialogues resemble more spoken informal discourse than written formal discourse. The difference in number of occurrences per I000 words between the Wizard-of-OZ condition and the Informal Spoken condition is much less than the same difference between the Wizard-of-OZ and the Formai Written conditions.</Paragraph>
    <Paragraph position="1">  Short, simple (g5% of our utterances were simple), active sentences, with few coordinations, few subordinations, few relative clauses, Few nominaiizations, (and deletion of determiners and unmarked agreement, see the section on Ungrammaticalities) characterize the language in typed user-adviser dialogues observed in our study. These same features are features of unplanned language, which are atso features or child language, which are also features of language produced under real-time production constraints (Ochs, 1079; Givon, 1970).</Paragraph>
    <Paragraph position="2">  While formality and completeness of typed user-adviser dialogues resemble more Formal Written language, the general syntactic features of typed user-adviser dialogues resemble more Informal Spoken language. Formality and completeness appear to be independent properties of users' language from the general syntactic features, possibly planned independently.</Paragraph>
    <Paragraph position="3"> More important for the design of naturM language interfaces, the observation that typed user-adviser dialogues resemble language produced under real-time production constralnts indicates that users are strained by typing utterances to request help to perform a primary task. This constrains the usability of natural language interfaces as interfaces to advisory systems. One needs to identify the conditions under which the benefits of obtaining help outweight the costs of typing in utterances to determine when natural language interfaces are effective interfaces to advisory systems. On the other hand, the natural restrictions on the language produced by the users appear generalizable to any situation where real-time production constraints exist, of which, we believe, any typed interaction to an advisory system for the purpose or performing a primary task is an instance.</Paragraph>
    <Paragraph position="4"> Features Due Specifically to the User-Advlslng Appllcatlon As can be seen in Table 3, there are less imperatives in user-advising dialogues because the user cannot request the adviser to perform a statistical operation. Moreover, we also observe a goal-directed language with frequent to infinitives (I want/need to ...) and to purpose clauses (What is the command to compute ...), much more frequent than in Informal Spoken or Formal Written languages. We believe this is the only feature that appears to be specific to the advisory application, as opposed to be specific to communications under real*time constraints. However, the goal-directedness of the language may be specific to advisory systems for procedural tasks as opposed to more generaln information retrieval tasks. Of course, we are here excluding lexical restrictions because they are expected and uninteresting and syntactic-semantic co-restrictions because of the desire for  In our study, users produced mostly very simple sentence constructions, as if under real-time production constraints (e.g., users' utterances were short and 95% of them were simple (see the section on General Syntactic Features)).</Paragraph>
    <Paragraph position="5"> Nevertheless, very few pronouns occurred, 3% of utterances contained pronouns, similar to what was found in Formal Written Language, Human-Computer dialogues, and in Cohen, Pertig, ,~ Start (1982) in their typed terminal-to-terminal condition. This is surprising because pronouns are very short to type. However, there were very frequent complex nominals with prepositional phrases (e.g., a record of tAc li~ting of the names of the features). At least 50,C/o of the ,:set-adviser utterances had one or more prepositional phrases..-ks can be seen in Table 4, most of the structurally ambiguous prepositional attachments arc to NPs, in fact, mostly to the most contiguous/nearest NP. So, users prefer longer to type complex nominals with explicit relations be* tween contiguous NPs over faster to type pronouns, even though there is evidence that they are operating under real~.ime production constraints. Because pronominal noun phrases (and also deictic expressions) are so rare, it appears that users rely little on spatial context (i.e., the screen), linguistic context (i.e., the utterances produced so far), and task context (i.e., statistical commands typed so far) in producing referring expressions. One interpretation of this finding is that users believe that there is poor shared context between user and adviser when they do not share physical context (as in Formal Written language) or do not know the linguistic capabilities of the conversant (a~ in Human-Computer dialogues). So, while in unplanned discourse speakers rely more on the context to express propositions and use more pronouns than in planned discourse (Ochs, 1979) and while user-adviser dialogues exhibit many features of unplanned discourse, users did not capitalize on context in producing referring expressions. It appears that the referential functions in language can be planned independently of and are not necessarily subject to the same real-time production constraints than the predicative and other functions of language. Again we are finding that typed user-adviser dialogues have some features of planned, Formal Written language but also have features of unplanned, Informal Spoken language.</Paragraph>
    <Paragraph position="6">  Nevertheless, not only are most prepositional attachments to NPs to create precise description of objects, they are mostly to the most contiguous NP. This observation suggests that real-time production constraints nevertheless play some role in the production of referential expressions. Users appear to minimize resources allocated to the production of referentiM expressions by reducing short-term memory load by attachments to the lowest, most recent NF.</Paragraph>
    <Paragraph position="7"> This interpretation is supported by studies that show that it is easier to process right-branching structures than left-branching ones (Yngve, 1060).</Paragraph>
    <Paragraph position="8"> The finding that most prepositional phrases attach to NPs rather than VPs and moreover attach most often to the lowest, nearest NP is important for the semantic interpretation of sentences because of the combinatorial explosion of possible attachments of prepositional phrases.</Paragraph>
  </Section>
class="xml-element"></Paper>