<?xml version="1.0" standalone="yes"?>
<Paper uid="P85-1020">
  <Title>MOVEMENT IN ACTIVE PRODUCTION NETWORKS</Title>
  <Section position="1" start_page="0" end_page="0" type="metho">
    <SectionTitle>
MOVEMENT IN ACTIVE PRODUCTION NETWORKS
Mark A. Jones
Alan S. Driacoll
AT&amp;T Bell Laboratories
</SectionTitle>
    <Paragraph position="0"/>
  </Section>
  <Section position="2" start_page="0" end_page="0" type="metho">
    <SectionTitle>
ABSTRACT
</SectionTitle>
    <Paragraph position="0"> We describe how movement is handled in a class of computational devices called active production networks (APNs). The APN model is a parallel, activation-basod framework that ha= been applied to other aspects of natural language processing. The model is briefly defined, the notation and mechanism for movement is explained, and then several examples are given which illustrate how various conditions on movement can naturally be explained in terms of limitations of the APN device.</Paragraph>
  </Section>
  <Section position="3" start_page="0" end_page="0" type="metho">
    <SectionTitle>
I. INTRODUCTION
</SectionTitle>
    <Paragraph position="0"> Movement is an important phenomenon in natural languages. Recently, proposals such as Gazdar's dcrivod rules (Gazdar, 1982) and Pereira's extraposition grammars (Pereirao 1983) have attemptod to find minimal extensions to the context-free framework that would allow the description of movement. In this paper, we describe a class of computational devices for natural language processing.</Paragraph>
    <Paragraph position="1"> called active production networks (APNs), and explore how certain kinds of movement are handled. In particular.</Paragraph>
    <Paragraph position="2"> we are concerned with left extraposition, such as Subject-auxiliary Inversion. Wh-movement, and NP holes in relative clauses, in these cos*s, the extraposod constituent leaves a trace which is insertod at a later point in the processing. This paper builds on the research reported in Jones (1983) and Jones (forthcoming).</Paragraph>
  </Section>
  <Section position="4" start_page="0" end_page="0" type="metho">
    <SectionTitle>
7,. ACTIVE PRODUCTION NgrwoPJ~
7..1 Tim i~vk~
</SectionTitle>
    <Paragraph position="0"> Our contention is that only a class of parallel devices will prove to be powerful enough to allow broad contextual priming, to pursue alternative hypotheses, and to explain the paradox that the performance of a sequential system often degrades with new knowledge, whereas human performance usually improves with learning and experience. = There are a number of new parallel processing (connection* st) models which are sympathetic to this view--Anderson (1983). Feldman and Ballard (1982), Waltz and Pollack (1985). McClelland and Rumelhart (1981, 1982), and Fahlman. Hinton and Sejnowski (1983).</Paragraph>
    <Paragraph position="1"> Many of the connection*st models use iterative relaxation techniques with networks containing excitatory and inhibitory links. They have primarily been used as best-fit categorizers in large recognition spaces, and it is not yet clear how they will implement the rule-governed behavior of parsers or problem solvers. Rule-based systems need a strong notion of an operating state, and they depend heavily on appropriate variable binding schemes for operations such as matching (e.g.. unification) and recurs*on.</Paragraph>
    <Paragraph position="2"> The APN model directly supports a rule-based interpretation, while retaining much of the general flavor of I. 1&amp;quot;be htmmm li~ity to L:mrfofm mmlpatztmmtlly e*patm,m opmltmm =alia s  ~y ~, imt'alkd loud,mum remforou this bC/fid.</Paragraph>
    <Paragraph position="3"> connection*sin. An active production network is a ruleoriented, distributed processing system based on the following principles: 1. Each node in the network executes a uniform activa- null tion algorithm and assumes states in response to message (,such as expectation, inhibition, and activation) that arrive locally; the node can, in turn, relay messages, initiate messages, and spawn new instances to process message activity. Although the patterns that define a node's behavior may be quite idiosyncratic or spocializod, the algorithm that interprets the pattern is the same for each node in the network.</Paragraph>
    <Paragraph position="4"> 2. Messages are relatively simple. They have an associated time, strength, and purpose (e.g., to post an expectation). They do not encode complex structures such as entire binding lists, parse trees, feature lists, or meaning representations, z Consequently, no structure is explicitly built; the &amp;quot;result&amp;quot; of a computation consists entirely of the activation trace and the new state of the network.</Paragraph>
    <Paragraph position="5"> Figure I gives an artificial', but comprehensive example of an APN grammar in graphical form. The grammar generates the strings--a, b. acd. ace. bed. bee. fg and gland illustrates mapy of the pattern language features and grammar writing paradigms. The network responds to $ourcex which activate the network at its leaves. Activation messages spread '*upward&amp;quot; through the network. At conjunctive nodes (seq and and), expectation messages are posted for the legal continuations of the pattern; inhibition messages are sent down previous links when new activations are recorded.</Paragraph>
    <Paragraph position="7"> In parsing applications, partially instantiatcd nodes are viewed as phrase structure rules whose next constituent is expected. The sources primarily arise from exogenous</Paragraph>
  </Section>
  <Section position="5" start_page="0" end_page="162" type="metho">
    <SectionTitle>
2. For * sit'tatar C/oaaectioaett vnew, ~ FC/ldman sad B#llard (1982) or
</SectionTitle>
    <Paragraph position="0"> Waltz ted Pollack (198S). A compemoa or markor patuns, value Imaan I *ad uoreltricted melmzle pinball =yttm=t= i= ipvea ia Fahlmnm, Hlalal lad Scjnowl~ (IgS)).  strobings of the network by external inputs. In generation or problem solving applications, partially instantiated nodes are viewed as partially satisfied goals which have out.standing subgoaLs whine solutions are de=ired. The source= in this case are endogenously generated. The compatibility of the=e two views not only allows the same network to be used for both parsing and generation, but also permits procesu~ to share in the interaction of internal and external sources of information. This compatibility, somewhat surprisingly, turned out to be crucial to our treatment of movement, but it is aLso clearly desirable for other aspects of natural language processing in which parsing and problem solving interact (e.8., referenco resolution and inferen(~P.). null Each node in an APN is defined by a pattern, written in the pattern language of Figure 2. A pattern describes the me=age= to which a node rmponds, and the new message= and internal state= that are produced. Each subpattern of the form ($ v binding-put) in the pattern for node N is a variable binding site; a variable binding takes place when an instance of a node in binding-gat activates a reference to variable v of node N. Implicitly, a pattern defines the set of state= and. state transitions for a node.</Paragraph>
    <Paragraph position="1"> The ? (optiouality), + (repetition) and * (optional repetition) operators do not extend the expressiveness of the language, but have been added for convenience. They can be replaced in preprocessin8 by equivalent expre&amp;sions, j Formal semantic definitions of the m_~_~$e passing behavior for each primitive operator have been specified.</Paragraph>
    <Paragraph position="2">  An important distinction that the pattern language makes is in the synchronicity* of activation signals. The pattern (and ($ vl X) ($ v2 \]'3) require= that the activation from X and F emanate from distinct network sources, while the pattern ($ v (and X I&amp;quot;3) insists that instances of X and Y are activated from the same source. In the \]. The enact chore= o( cq~s'acors in the pattern tan|up it t matewhat ~at= mine from the =!~=m~attma of the APN maciaa~.</Paragraph>
    <Paragraph position="3">  4, -r~ C/nulreat APN model allocate= ~ telueatmUy. The ten= $yllgiteomlclly reflC~lt thl fact thll t\[~ ~ kicl~Uly o4 r t~ i~ m~se= can be Ioc~y COmlm '.,,I f~m tlm=r tiuw ~f ~ TI~ u kin |u the ,ctJvuua= pmau= rims \[=== a~ugb to coacli~aa the network bmmi, mai my, m0 scuvatmm. Alua'aaUvety, a,:Uvalma melal~ covid emV tl~ mmrC/~ ideatiW =t as a4di,t*...-t l~ram, et~n. ia tl~ csm. m=Jme aeuntiom cam a*su.hq~ ~ at t t'' prom t'~h, ~ e( tit iaaemmltal cxp~C/ume ~mvtm,,_,~._ F.= re~l~iy illlequndeut i ,i.. o,m'lap may nm po~ a p~Vlem.</Paragraph>
    <Paragraph position="4">  graphical representation of an APN, synchrony is indicated by a short tail above the subpattern expression; the definition of U in Figure I illustrates both conventions: (and ($ vl (and TI)) ($ v2 S)).</Paragraph>
    <Section position="1" start_page="161" end_page="162" type="sub_section">
      <SectionTitle>
2.3 Am F..~m~
</SectionTitle>
      <Paragraph position="0"> Figure 3 shows the stages in parsing the string acd. An exogenous source Exog-srcO first activates a, which is not currently supported by a source and, hence, is in an inactive state. The activation of an inactive or inhibited node give= rise to a new instance (nO) to record the binding.</Paragraph>
      <Paragraph position="1"> The instance is effectively a new node in the network, and derives its pattern from the spawning node. The activation spreads upward to the other instances shown in Figure 3(a). The labels on each node indicate the current activation level, repreu:nted as an integer between 0 and 9, inclusive.</Paragraph>
      <Paragraph position="3"> (a) trace structure after a</Paragraph>
      <Paragraph position="5"> (c) trace structzure after acd \[~ple 3, Stalp=l in Parsing acd  The activation of a node causes its pattern to be 4re)instantiated and a variable to be (re)bound. For exampie. in the activation of RO, the pattern (seq ($ vi Q) (5 v2 c'9) is replaced by (seq ($ vi (or Q QO)) ($ v2 c)). and the variable vl is bound to (20. For simplicity, only the active links are shown in Figure 3. RO posts an expectation message for node C which can further its pattern. The source Exog-secO is said to be supporting the activation of nodea nO. QO. RO and PO above it, and the expectations or inhibitions that are generated by these nodes. For the current paper we will assume that exogenous sources remain fully on for the duration of the sentenco, s In Figure 3(b), another exogenous source Exog-srcl activates c, which furthers the pattern for RO. RO sends an inhibition message to QO, posts expectations for S, and relays an activation message to P0, which rebind~ its variable to RO and a~umes a new activation value. Figure 3(c) shows the final situation after d has been activated. The synchronous conjunction of SO is satisfied hy TO and dO. RO is fully satisfied (activation value of 9), and PO is re-satisfied.</Paragraph>
      <Paragraph position="6"> 1,4 Gramm~ Writbql P~Ulpm The APN in Figure I illustrates several grammar writing paradigms. The situation in which an initial prefix string (a or b) satisfies a constituent (P), but can be followed by optional suffix strings (cd or ce) occurs frequently in natural language grammars. For example, noun phrase heads in English have optional prenominal and postnominal modifiers. The synchronous disjunction at P allows the local role of a or b to change, while preserving its interpretation as part of a P. It is also simple to encode optional prefixes.</Paragraph>
      <Paragraph position="7"> Another common situation in natural language grammars is specialization of a constituent based on some interhal feature. Noun phrases in English, for exampl(c), can be specialized hy case; verb phrases can be specialized as participial, tensed or infinitive. In Figure l, node S is a spe. cialization which represents &amp;quot;Ts with d-ness or e-ness, but not f-heSS.'&amp;quot; The specialization is constructed by a synchronous conjunction of features that arise from subtrees somewhere below the node to be specialized.</Paragraph>
      <Paragraph position="8"> The APN model also provides for node outputs to he partitioned into independent classes for the purl~sC/~ ,~)f the activation algorithm. The nodes in the classes form levels in the network and represent orthogonal systems of classification. The cascading of expectations from dilfcrent I~els can implement context-sensitive behaviors such as feature agreement and s':mantic sclectionai restrictiops.</Paragraph>
      <Paragraph position="9"> This is described in Jones (forthcoming). In the next section, we will introduce a grammar writing paradigm to represent movement, another type of non..context-freC/  behavior.</Paragraph>
      <Paragraph position="10"> $. It is interertins to sp~'ulatc: on the oOm~lUamC~ o( vsr~w relauua~q of ~hiu C/al~m~l~Oe. Fundam,mt~l limitatmm in the allocatm of ~ may be reJalod to limiuUmna in sluart term memory (~r buff're space in</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>