<?xml version="1.0" standalone="yes"?>
<Paper uid="P98-1087">
  <Title>A Connectionist Architecture for Learning to Parse</Title>
  <Section position="3" start_page="0" end_page="0" type="intro">
    <SectionTitle>
1 Introduction
</SectionTitle>
    <Paragraph position="0"> Connectionist networks are popular for many of the same reasons as statistical techniques. They are robust and have effective learning algorithms. They also have the advantage of learning their own internal representations, so they are less constrained by the way the system designer formulates the problem. These properties and their prevalence in cognitive modeling has generated significant interest in the application of connectionist networks to natural language processing. However the results have been disappointing, being limited to artificial domains and oversimplified subproblems (e.g. (Elman, 1991)). Many have argued that these kinds of connectionist networks are simply not computationally adequate for learning the complexities of real natural language (e.g. (Fodor and Pylyshyn, 1988), (Henderson, 1996)).</Paragraph>
    <Paragraph position="1"> Work on extending connectionist architectures for application to complex domains such as natural language syntax has developed a theoretically motivated technique called Temporal Synchrony Variable Binding (Shastri and Ajjanagadde, 1993; Henderson, 1996). TSVB allows syntactic constituency to be represented, but to date there has been no empirical demonstration of how a learning algorithm can be effectively applied to such a network. In this paper we propose an architecture for TSVB networks and empirically demonstrate its ability to learn syntactic parsing, producing results approaching current statistical techniques.</Paragraph>
    <Paragraph position="2"> In the next section of this paper we present the proposed connectionist architecture, Simple Synchrony Networks (SSNs). SSNs are a natural extension of Simple Kecurrent Networks (SRNs) (Elman, I99I), which are in turn a natural extension of Multi-Layered Perceptrons (MLPs) (Rumelhart et al., 1986). SRNs are an improvement over MLPs because they generalize what they have learned over words in different sentence positions. SSNs are an improvement over SKNs because the use of TSVB gives them the additional ability to generalize over constituents in different structural positions. The combination of these generalization abilities is what makes SSNs adequate for syntactic parsing.</Paragraph>
    <Paragraph position="3"> Section 3 presents experiments demonstrating SSNs' ability to learn syntactic parsing. The task is to map a sentence's sequence of part of speech tags to either an unlabeled or labeled parse tree, as given in a preparsed sample of the Brown Corpus. A network input-output format is developed for this task, along with some linguistic assumptions that were used to simplify these initial experiments. Although only a small training set was used, an SSN achieved 63% precision and 69% recall on unlabeled constituents for previously unseen sentences. This is approaching the 75% precision and recall achieved on a similar task by Probabilistic Context Free Parsers (Charniak, forthcoming), which is the best current method for parsing based on part of speech tags alone. Given that these are the very first results produced with this method, future developments are likely to improve on them, making the future for this method very promising.</Paragraph>
  </Section>
class="xml-element"></Paper>