<?xml version="1.0" standalone="yes"?>
<Paper uid="W98-1202">
  <Title>Natural Language Learning by Recurrent Neural Networks: A Comparison with probabilistic approaches</Title>
  <Section position="7" start_page="6" end_page="6" type="concl">
    <SectionTitle>
5. Conclusions
</SectionTitle>
    <Paragraph position="0"> We have described results for the training of Elman and RCC networkson a natural language task. The task is to predict the part-of-speech category of the next word in a sentence given the category of the current word as input.</Paragraph>
    <Paragraph position="1"> The Elman network appears to be a more useful model for this one-step-look-ahead task than the RCC network.</Paragraph>
    <Paragraph position="2"> Elman networks are statistical learners and we have shown that network learning can be interpreted in t=rrts of learning n-gram statistics. However because network learning is driven by minimisation of predictive error, longer sequences having high frequency bias learning more than infrequently occurring short sequences.</Paragraph>
    <Paragraph position="3"> The sequences correctly learned by the Elman network included some that were not predicted by trigram probabilities, evidence that the network was using the previous three or more inputs as context for prediction.</Paragraph>
    <Paragraph position="4"> Prediction uncertainty was highest when the input was the first verb category in the sentence, possibly Towsey, Diederich, Schellharnmer, Chalup, Brugman 9 Natural Language Learning by Recurrent Neural Nets consistent with the important role that the verb plays in the syntactic structure of a sentence.</Paragraph>
  </Section>
class="xml-element"></Paper>