<?xml version="1.0" standalone="yes"?>
<Paper uid="H89-2078">
  <Title>White Paper on Natural Language Processing</Title>
  <Section position="1" start_page="0" end_page="486" type="metho">
    <SectionTitle>
SRI International
</SectionTitle>
    <Paragraph position="0"> We take the ultimate goal of natural language processing (NLP) to be the ability to use natural languages as effectively as humans do. Natural language, whether spoken, written, or typed, is the most natural means of communication between humans, and the mode of expression of choice for most of the documents they produce. As computers play a larger role in the preparation, acquisition, transmission, monitoring, storage, analysis, and transformation of information, endowing them with the ability to understand and generate information expressed in natural languages becomes more and more necessary. Some tasks currently performed by humans cannot be automated without endowing computers with natural language processing capabilities, and these provide two major  challenges to NLP systems: 1. Reading and writing text, applied to tasks such as message routing, abstracting, monitoring, summarizing, and entering information in databases, with applications, in such areas as intelligence, logistics, office automation, and libraries. Computers should be able to assimilate and compose extended communications.</Paragraph>
    <Paragraph position="1"> 2. Translation, of documents or spoken language, with applications, in such areas as in science, diplomacy,  multinational commerce, and intelligence. Computers should be able to understand input in more than one language, provide output in more than one language, and translate between languages.</Paragraph>
    <Paragraph position="2"> The dominance of natural language as a means of communication in a broad range of interactions among humans suggests that it would be an attractive medium in human-computer interaction as well. The case is particularly strong where the environment precludes the use of keyboard, display, and mouse, so that spoken natural language is almost the only alternative. However, speech recognition alone will not suffice in these settings. Words and phrases must be parsed and interpreted so that their intended meaning (as command, query, or assertion) may be determined and an appropriate response formulated and expressed.</Paragraph>
    <Paragraph position="3">  Even where other devices are available, the artificial languages they give the user access to -- menu and icon selection, and programming, command, and database query languages -- are limited. Menus and icons make it easy to present the user with the available options at any time, but they constrain the user to operating on visible objects only. It is also awkward or impossible to operate on sets of objects selected by complex properties (e.g., &amp;quot;Send all the C3 ships with RRI radar to the nearest port&amp;quot;). Programming and other &amp;quot;linear&amp;quot; artificial languages offer well-defined control structures but axe more difficult to learn, and do not take advantage of pointing to objects on the screen. Moreover, they typically require substantial knowledge of underlying representations, and they offer the user little guidance as to what can be done next. In fact, interaction with computers in any artificial language places on the user most of the burden of discovering how to express in the language the commands necessary to achieve the desired objective. It is natural for humans using natural languages to state complex conditions, to integrate these with pointing, and to negotiate how a task could and should be done. Natural language should, however, be seen as a powerful addition to the repertoire of methods for human-machine interaction, and not as a replacement for those methods.</Paragraph>
    <Paragraph position="4"> Thus, the third major challenge of natural language processing is 3. Interactive dialogue, allowing humans simple, effective access to computer systems, using natural language and other modalities, for problem-solving, decision-making, and control. Application areas include database access, command and control, factory control, office automation, logistics, and computer-assisted instruction. Human-machine interaction should be as natural, facile, and multi-modal as interaction among humans.</Paragraph>
    <Paragraph position="5"> While it will be quite some time before systems meet these challenges with the depth and flexibility that humans bring to them, useful shorter term goals of economic value have been and can continue to be met. These include database access systems, multi-modal interfaces to expert systems and simulators, processing of text (e.g., for database update), and semi-automatic machine translation systems.</Paragraph>
    <Paragraph position="6"> Virtually all developments necessary to long- and short-term progress toward these goals will be relevant to the transition from speech recognition systems to spoken language systems (SLS), which is the subject of a separate report to this committee. However, in this report we will not have anything ~rther to say specifically about SLSs or their attendant problems in speech signal processing, 1.2. Barriers to Progress 1.2.1. Success and Limitations Thus Far NLP systems perform three related functions: analysis (or interpretation) of the input, mapping it into an expression in some meaning representation language (MRL); reasoning about the interpretation to determine the content of what should be produced in response to the user, perhaps accessing information in databases, expert systems, etc: and finally, generation of a response, perhaps as a natural-language utterance or text. In translation systems, the &amp;quot;response&amp;quot; is in a language different from the input language.</Paragraph>
    <Paragraph position="7"> The most visible results in NLP in the last five years are several commercially available systems for database question-answering. These systems, the result of transferring technology developed in the 1970s and early 1980s, have been successfully used to improve productivity by replacing fourth-generation database query languages. The following case study illustrates their capabilities: with 8 person weeks, one of these systems was ported to a Navy relational database of 666 fields (from 75 relations) with a vocabulary of over 6,000 (root) words.  Queries from a new user of one of these systems are estimated to succeed 60 to 80% of the time; with use of the system users naturally and automatically adjust to the data that is in the database and to the limits of the language understood by the system, giving a success rate of 80 to 95%, depending on the individual.</Paragraph>
    <Paragraph position="8"> The success of these systems has depended on the fact that sufficient coverage of the language is possible with relatively simple semantic and discourse models. The semantics are bounded by the semantics of the relations used in databases and the fact that words have a restricted number of meanings in one domain. The discourse model for a query is usually limited to the table output of the previous answer and the noun phrases mentioned in the last few queries.</Paragraph>
    <Paragraph position="9"> The limitations of today's practical language processing technology are summarized as follows: * Domains must be narrow enough that the constraints on the relevant semantic concepts and relations can be expressed using current knowledge representation techniques, primarily in terms of types and sorts. Processing may be viewed abstractly as the application of recursive tree rewriting rules, including filtering out trees not matching a certain pattern.</Paragraph>
    <Paragraph position="10"> * Handcrafting is necessary, even in the grammatical components of systems, the component technology that exhibits least dependence on the application domain. Lexicons and axiomatizations of critical facts must be developed for each domain, and these remain time-consuming tasks.</Paragraph>
    <Paragraph position="11"> * The user must still adapt to the machine, but, as the products testify, can do so effectively.</Paragraph>
    <Paragraph position="12"> * Current systems have limited discourse capabilities which are almost exclusively handcrafted. Thus current systems are limited to viewing interaction, translation, and writing and reading text as processing a sequence of rather isolated sentences. Consequently, the user must adapt to such limited discourse.</Paragraph>
    <Paragraph position="13"> 1.2.2. Status of Evaluation in NLP Natural language processing combines basic scientific challenges with diverse technological needs.</Paragraph>
    <Paragraph position="14"> Evaluating progress in scientific challenges for NLP is a multifaceted issue, perhaps akin to the problem of evaluating progress in the field of programming languages. While certain aspects of progress in that field can be quantified, this is generally not the case; it is hard to quantify how much better one programming language is than another, for example. Nevertheless, there is still reason to believe that scientific issues are becoming better defined and that progress is being made.</Paragraph>
    <Paragraph position="15"> On the other hand, evaluation metrics for technological advances based on the science are being developed and applied. In machine translation, measurements have evolved further than in other areas, and they encompass both range of application (what kind of texts in what domain can be translated) and accuracy of the translation (percentage of sentences that remain semantically invariant, and of those, the percentage that are stylistically acceptable). Natural language interfaces can be evaluated in terms of their habitability; that is, how well and how fast can a user get the task accomplished? How often do first phrasings work? Do subsequent human-machine clarification, or focused rephrasing, yield success? These criteria, however, evaluate performance of the technology in a task domain, rather than the underlying science, independent of task.</Paragraph>
    <Paragraph position="16"> It behooves the field to continue refining and applying qualitative as well as quantitative measures of progress in the NLP task domains, and to consider the issue of how to measure and evaluate research in the scientific core, if that is to be done any differently from the standard measures of publication and scholarship of most established scientific fields.</Paragraph>
    <Paragraph position="17">  Given the limitations of the current technology discussed in Section 1.2.1, fundamental scientific problems in the following three broad areas should be addressed: Adequate theories of semantics and discourse. These theories should apply to both generation and interpretation of interactive dialogue and text and must support communication across a range of domains of discourse. I.n semantics, this means at a minimum that we must have a semantics of words (a lexical semantics) that is independent of the domain and of the application, and that the meaning of a word is easily (semi-automatically) related to the concepts of particular domains and applications. Accounts of the combined use of linguistic and non-linguistic (e.g., pointing) intbrmation in interactive dialogues should also be developed. The particular styles of certain sublanguages, e.g., Army Operations Orders, should also be accommodated.</Paragraph>
    <Paragraph position="18"> Acquisition of information necessary to understanding and creating communication. This includes both linguistic (e.g., words and grammatical forms) and non-linguistic information (e.g., the semantics of icons) used by real users performing real tasks in a given domain of discourse.</Paragraph>
    <Paragraph position="19"> A calculus of partial information. In both single- and multiple-sentence texts and dialogues, generation and interpretation requires combining information from a number of sources, such as morphological, syntactic, semantic, pragmatic, and prosodic information. This is particularly true with novel, errorful, and incomplete expressions. Current systems are limited in the kind of information-they bring to hear on interpretation, and the processes by which they do so. Both logical and statistical methods may be further investigated.</Paragraph>
    <Paragraph position="20">  Lack of Leverage. Building an NL system requires an extensive effort over several years. Most researchers lack the resources to produce a complete system and lack access to state-of-the-art software for some components (e.g., parsers, large grammars, task-specific lexicons, knowledge representation systems, semantic interpreters). Having such components would let them concentrate their efforts on novel work, demonstrate a complete system, and test individual component theones. Maximal sharing of components requires that a few common tasks be selected by the community and that appropriate backend systems (databases, simulators, expert systems, etc.) be made widely available. Leverage can he further increased by development and support of key NL components. Collection and dissemination of large linguistic data sets will support development of broad-coverage grammars, better lexicons, systematic evaluation procedures, and statistical measures.</Paragraph>
    <Paragraph position="21"> Funding. Overall funding for NLP has been strong from 1984 through 1988. However, DARPA funding in the last several years has increasingly emphasized technology transfer and near-term results. Although this emphasis has had some positive as well as negative results, the overall trend is cause for concern. On the positive side, the focus on shorter term performance has forced the community to focus on the development of prototypes addressing specific tasks in specific domains and to think about evaluation methods and resource sharing. On the negative side, it has left little room for developing the theoretical basis of the next generation of systems. Some of this responsibility has been taken on by other sources (notably the Systems Development Foundation and Japanese industry), but support from the former is coming to an end, and there are obvious reasons for not wanting to depend too heavily on the latter. Given these factors, we have serious concern for future levels of basic research funding. Training of Researchers. Researchers in NLP need a broad exposure to AI, computer science, linguistics, logic, and increasingly to probability and statistics. It is important that the funding of research projects, in and out of universities, allow for student participation.</Paragraph>
    <Section position="1" start_page="484" end_page="486" type="sub_section">
      <SectionTitle>
1.3. Anticipated Developments
</SectionTitle>
      <Paragraph position="0"> During the next decade, we anticipate several scientific breakthroughs which shouM bring about impact noticable to the user community.</Paragraph>
      <Paragraph position="1">  Within the next 3 to 10 years we foresee the following scientific breakthroughs:  * Architectures that support coordinating syntactic, semantic, and pragmatic constraints, that deal with partial information, and that understand novel, errofful, and vague forms. * A robust, task-independent, compositional semantics, including more thorough treatment of problems relevant to major application areas, such as time and tense, adverbs and adjectives, conjunctions and ellipsis.</Paragraph>
      <Paragraph position="2"> * Automatic acquisition of substantial glammars and lexicons.</Paragraph>
      <Paragraph position="3"> * Parallel algorithms for key processes.</Paragraph>
      <Paragraph position="4"> * Computational models of discourse structure and speaker intention adequate to support dialogue participation and text generation.</Paragraph>
      <Paragraph position="5"> 1.3.2. Technology Transfer Existing laboratory prototypes coupled with the scientific breakthroughs projected above suggest that in the next decade a new generation of systems, having the properties below, will be available: * Text analysis systems for automatic database update, m restricted domain areas. * Interactive problem-solving systems combining NL, pointing, and graphical access to several target systems (e.g., databases, simulators, expert systems); exhibiting extended conversations including clarifications, suggestions, and confirmations; and allowing rapid, low-cost portability from one (constrained) application domain to another.</Paragraph>
      <Paragraph position="6"> * Language generation systems producing extended texts in limited applications (e.g., summarization of databases or output and explanations of expert systems' decisions).</Paragraph>
      <Paragraph position="7"> 2. Background 2.1. Current Assessment  Products. In the decade since 1978, at least eight commercial products for natural language access to databases have been released. Two message processing systems are m daily use, one for the U.S. Coast Guard. In the U.S. alone, four companies offer machine translation systems.</Paragraph>
      <Paragraph position="8"> Limitations of current systems. The limitations of the current technology, described in Section 1.2.1. of this paper, can be illustrated by considering NL access to databases, the application that has probably received more support than any other in the U.S. in the last ten years. The nature of the task limits the range of inputs the system can expect to see and the semantic distinctions that need to be reflected m the MRL. Reasoning m relational databases is limited to the operations of relational algebra on purely extensional information, so certain concepts, such as tense and modality, need not be reflected in the MRL either. Limitations on the content of a database can  guarantee that certain interpretation ambiguities will not arise, or that they can often be resolved by simple means. In a geographical database, occurrences of the noun &amp;quot;bank&amp;quot; as a financial institution probably never need be considered at all in interpreting &amp;quot;What are the cities on the left bank of the Rhone?&amp;quot;. If countries but not mountains have populations, then in &amp;quot;What is the population of Kenya?&amp;quot;, Kenya means the country, not the mountain. By assuming the user will adapt to the system and that NL can substitute for an artificial language, the NL interface treats each question in isolation with only a very general, weak notion of tim goal in an utterance. The availability of these kinds of restrictions has aUowed NL database query systems to be successful using relatively simple frameworks in which to encode the necessary knowledge sources {grammars, type and sortal information, lists of mentioned entities) applied in relatively simple ways (parsing, recursive tree transformations). This is not to minimize the effort required to build the grammar, semantic model, etc. for a particular application. The size of the vocabulary per se is not a limiting factor, though it does impact the initial cost of bringing up the NLP. Rather what is limiting is the number of word senses per word in the vocabulary, whether the language involves substantial intersentential effects (discourse structure), and whether the underlying semantics is richer than that of relational databases.</Paragraph>
      <Paragraph position="9"> Scientific progress. The scientific progress of the last 10 years can be described in terms of traditional linguistic areas and in terms of task areas.</Paragraph>
      <Paragraph position="10"> The main development in syntax has been the shift from grammars including procedural constructs to ones expressed purely declaratively. In contrast to context-free grammars, which use atomic symbols only, these so called unification grammars use complex terms instead, with term unification instead of equality checking as the main operation. &amp;quot;nais has allowed for the use of the same grammars with a range of algorithms, both sequential and parallel, for analysis and generation. Grammar development tools have been written in a variety of unification-based frameworks, and widely distributed. Unification grammars are currently being used to apply syntactic (and some semantic) constraints in speech recognition. Within the family of unification grammars lie the &amp;quot;mildly context-sensitive grammars&amp;quot; -- a class that properly contains the context-free grammars, and allows the expression of observed syntactic constraints not expressible in CFGs, but whose recognition problem is polynomial-time computable.</Paragraph>
      <Paragraph position="11"> In semantics, the major aspects of the contribution of sentence structure to meaning are understood and implemented. First steps toward automatically extracting aspects of lexical meaning from machine-readable dictionaries have been taken.</Paragraph>
      <Paragraph position="12"> Understanding and generating connected sentences introduces questions such as how texts and dialogues are structured; how this structure affects interpretation, particularly of referential expressions; how the beliefs, intentions, and plans of a speaker are conveyed by what is said and how they constrain what is meant, as well as what appropriate responses are. All these questions have been and continue to be investigated. Underlying logics and algorithms for reasoning about knowledge, belief, intention, and action have been proposed, as have initial computational models for discourse structure and methods for planning and plan recognition for discourse. Although progress continues to be made on query systems, substantial systems have been developed for other applications, all of which were unexplored 10 years ago. These include several text processing systems supported by DARPA's Strategic Computing Imtiative (SCI), ranging from systems with very detailed models of one domain, to more general ones adapted to several domains (e.g., Naval Casualty Reports, RAINFORMS, terrorist reports). Multi-media interactive problem solving systems have been developed for the environment of Navy Fleet Command Center decision making and for factory control. Language generation systems have been implemented to  generate multi-sentence explanations of expert system decisions, object or situation descriptions, and instructions from an expert.</Paragraph>
      <Paragraph position="13">  Historically, NLP's strongest interaction has been with other areas of artificial intelligence, e.g., with work in knowledge representation and planning. During the 1980s, collaboration with theoretical linguistics and cognitive science has been growing. Lack of widespread availability of high-performance, parallel computers thus far has limited the algorithms considered; however, efforts in parallelism, including work in connectionist neural network modeling, may grow in the next decade.</Paragraph>
      <Paragraph position="14"> Until two years ago, interaction with speech scientists had been minimal since the end of the DARPA Speech Understanding Program in 1976. Progress ha natural language processing should contribute directly to spoken language systems. This is true not only where understanding seems to be necessary, e.g., voice commands and requests, translation of speech, etc., but also in speech transcription. The error rate of speech recognition systems is directly correlated with the perplexity of the language to be recognized. Statistical language models in speech transcription have given the lowest perplexity thus far, and therefore, the best performance. Language processing techniques, whether supplemental or in place of current statistical models, offer the potential of providing even lower perplexity due to modeling both local and global constraints, as well as supporting speech applications other than transcription.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>