<?xml version="1.0" standalone="yes"?>
<Paper uid="C90-2038">
  <Title>Incremental Sentence Production with a Parallel Marker-Passing Algorithm</Title>
  <Section position="3" start_page="0" end_page="0" type="intro">
    <SectionTitle>
2 Basic Organization of the Model
</SectionTitle>
    <Paragraph position="0"> We use a hybrid parallel paradigm \[Kitano, 1989b\], which is an integration of a parallel marker-passing scheme and a connectionist network, as a basic algorithm. Five types of markers (two types for parsing, two other types for generation, and an another type for contextual priming) are passed around the memory network which represents knowledge from morphophonetic-level to discourse-level. A connectionist network performs sub-symbolic computations with a massive parallelism.</Paragraph>
    <Paragraph position="1"> Use of the hybrid parallel scheme on the memory network has its merit in exploring implicit parallelism in the process of natural language generation and parsing.</Paragraph>
    <Section position="1" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
2.1 The Memory Network
</SectionTitle>
      <Paragraph position="0"> The memory network incorporates knowledge from morphophonetics to plan hierarchies of each participant of a dialog. Each node is a type and represents either a concept (Concept Class node; CC) or a sequence of concepts (Concept Sequence Class node; CSC). Strictly speaking, both CC and CSC are a collection or family since they are, for the most part, sets of classes. CCs represent such knowledge as concepts (i.e. *Conference, *Event, *Mtrans-Action), and plans (i.e. *Declare-Want-Attend). CSCs represent sequences of concepts and their relations such as concept sequences 2 (i.e. &lt;*Conference *Goal-Role *Attend *Want&gt;) or plan sequences (i.e. &lt;*Declare-Want-Attend *Listen-Instruction&gt;) 3 of the two participants of the dialog. CSCs have an internal structure composed of a concept sequence, constraint equations, presuppositions, and effects. This internal structure provides our scheme with the capability to handle unification-based processing as well as case-based processing, so that typical criticisms against DMAP-type NLP \[Riesbeck and Martin, 1985\], such as weak linguistic coverage and incapability of handling linguistically complex sentences, do not apply to our model 4. Each type of node creates instances during parsing which are called concept instances (CI) and concept sequence instances (CSI), respectively. CIs correspond to discourse entities. They are connected through labelled links such as IS-A or PART-OF, and weighted links which form a connectionist network. CSIs record specific cases of utterances indexed into the memory network whereas ZConcept sequences are the representation of an integrated syntax/semantics level of knowledge in our model.</Paragraph>
      <Paragraph position="1"> 3This should not be confused with 'discourse segments' \[Grosz and Sidner, 1985\]. In our model, information represented in discourse segments is distributively incorporated in the memory network.</Paragraph>
      <Paragraph position="2"> 4Indeed, our model is substantially different from DMAP-type marker-passing or any other naive marker-passing models, because linguistic features are carried up by markers to conduct substantial linguistic analysis as well as case-based processing.</Paragraph>
      <Paragraph position="4"> CSCs represent generalized cases and syntactic rules. Use of cases for generation is one of the unique features of our model while most generators solely depend upon syntactic rules.</Paragraph>
    </Section>
    <Section position="2" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
2.2 Tile Markers
</SectionTitle>
      <Paragraph position="0"> A guided marker-passing scheme is employed for inference in the memory network. Basically, our model uses four types of markers. These markers are (1) activation markers, (2) prediction markers, (3) generation markers, and (4) verbalization markers.</Paragraph>
      <Paragraph position="1"> Activation Markers (A-Markers) are created based on the input of the source language. These are passed up through IS-A links and carry instance, features and cost. This type of marker is used for parsing.</Paragraph>
      <Paragraph position="2"> Prediction Markers (P-Markers) are passed along the conceptual and phonemic sequences to make predictions about which nodes are to be activated next. Each P-Marker carries constraints, cost, and the information structure of the utterance which is built incrementally during parsing.</Paragraph>
      <Paragraph position="3"> Generation Markers (G-Markers) show activation of nodes in the target language, and each contains a surface string, features, cost and an instance which the surface string represents. G-Markers are passed up through IS-A links.</Paragraph>
      <Paragraph position="4"> Verbalization Markers (V-Markers) anticipate and keep track of verbalization of surface strings. Final surface realizations, cost and constraints are carried by VMarkers. null Besides these markers, we assume Contextual Markers (C-Markers) \[Tomabechi, 1987\] which are used when a connectionist network is computationally too expensive. The C.-Markers are passed through weighted links to indicate contextually relevant nodes.</Paragraph>
    </Section>
    <Section position="3" start_page="0" end_page="0" type="sub_section">
      <SectionTitle>
2.3 A Baseline Algorithm
</SectionTitle>
      <Paragraph position="0"> Generally, natural language generation involves several stages: content deliniation, text structuring, lexical selection, syntactic selection, coreference treatment, constituent ordering, and realization. In our model, the content is determined at the parsing stage, and most other processes are unified into one stage, because, in our model.</Paragraph>
      <Paragraph position="1"> lexica~ item, phrase, and sentence are treated in the same mechanism. The common thrust in our model is the hypothesis-activation-selection cycle in which multiple hypotheses are activated and where one of them is finally selected. Thus, the translation process of our model is composed of processes of (1) concept activation, (2) lexical and phrasal hypotheses activation, (3) propositional content activation, (4) syntactic and lexical selection, and  (5) realization.</Paragraph>
      <Paragraph position="2"> 1. Concept Activation: A part of the parsing process as well as an initial process of generation. Individual concepts represented by CCs are activated as a result of parsing speech inputs. A-Markers are created and passed up by activating the concept.</Paragraph>
      <Paragraph position="3"> 2. Lexieal and Phrasal Hypotheses Activation: Hypotheses for lexicons and phrases which represent the activated concept are searched for, and G-Markers are created and passed up as a result of this process. Usually, multiple candidates are activated at a time.</Paragraph>
      <Paragraph position="4"> 3. Propositional Content Activation: A part of the parsing process by which propositional content of the utterance is determined.</Paragraph>
      <Paragraph position="5"> 4. Syntactic and Lexical Selection: Selection of one hypothesis from multiple candidates of lexical entries or phrases. First, the syntactic and semantic constraints are checked to ensure the correctness of the hypotheses, and the final selection is made using a cost/activation-based selection.</Paragraph>
      <Paragraph position="6"> 5. Realization: The surface string (which can be either a  sequence of words or a sequence of phonological signs) is formed from the selected hypothesis and scmt to the speech synthesis device.</Paragraph>
      <Paragraph position="7"> The movement of V-Markers is important in understanding our algorithm. First, a V-Marker is located on the first element of the CSC. When a G-Marker hits the element with the V-Marker, the V-Marker is moved to the next element of the CSC (figure la), and unification is performed to ensure syntactic soundness of the sentence. In figure lb, dl is a closed class lexical item s. When a G-Marker hits the first element, a V-Marker on the first element is moved to the third element by passing through the second element which is a closed class item. In this case, the element for the closed class item need not have a G-Marker. The lexical realization for the element is retrieved when the V-Marker passes through the element. In the case where the G-Marker hits an element without a V-Marker, the G-Marker is stored in the element. When another G-Marker hits the element with a V-Marker, the V-Marker is moved to the next element.</Paragraph>
      <Paragraph position="8"> Since the next element already has a G-Marker, the V-Marker is further moved to the subsequent element of the CSC (figure lc). Although, in most cases, a bottom up process by G-Markers handles generation process, there are cases where a bottom up process alone can not identify syntactic structure and lexieal items to express a given meaning. In such cases, a top-down process is invoked which identifies the best syntactic structure and lexieal items by searching downward from each element of the activated CSC. Each retrieval procedure is similar to the search of a closed class lexical item.</Paragraph>
      <Paragraph position="9"> There are cases in which an element oftheCSC is linked to other CSCs, and forms hierarchies of CSCs. Suppose each CSC represents a phrase structure nile, then the dynamically organized CSC hierarchy provides productive power so that various types of structures of complex sentences can be generated. In the hierarchy of CSCs, G-Markers are passed up when a CSC is accepted, and carry feature structures which represent mourning fragments expressed by the CSC. V-Markers are passed down to lower CSCs when an element is predicted, and impose constraints on each elements of the lower CSCs. The hierarchical organization of CSCs allows all types of tree expansions: upward, downward and insertion.</Paragraph>
      <Paragraph position="10"> Figure 2 shows an example of how an analysis tree can be constructed in our model. In this example, we assume Lexical-Functional Grammar (LFG) as a grammar formalism, and the order which conceptual fl'agments are given is based on an order that conceptual fragments can be identified when parsing a corresponding Japanese sentence incrementally. Notice that all three types of extensions are involved even in such a simple sentence.</Paragraph>
      <Paragraph position="11"> SClosed class lexical items refer to function words such as in, of, at in English and wo, ga, ni in Japanese. These words are non-referential and their number do not grow, whereas open class lexical items are mostly referential and their number grows as vocabulary expands.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>