<?xml version="1.0" standalone="yes"?>
<Paper uid="P98-2201">
  <Title>A Connectionist Approach to Prepositional Phrase Attachment for Real World Texts</Title>
  <Section position="7" start_page="1234" end_page="1235" type="concl">
    <SectionTitle>
5 Training and Experimental
</SectionTitle>
    <Paragraph position="0"> Results.</Paragraph>
    <Paragraph position="1"> 21418 examples of structures of the kind 'VB N1 PREP N2' were extracted from the Penn-TreeBank Wall Street Journal (Marcus et al. 1993). Word-Net did not cover 100% of this material. Proper names of people were substituted by the WordNet class someone, company names by the class business_organization, and prefixed nouns for their stem (co-chairman ---* chairman). 788 4-tuples were discarded because of some of their words were not in WordNet and could not be substituted. 20630 codified patterns were finally obtained: 12016 (58.25%) with the PP attached to N1, and 8614 (41.75%) to VB.</Paragraph>
    <Paragraph position="2"> We used the cross-validation method as a measure of a correct generalization. After encoding, the 20630 patterns were divided into three subsets: training set (18630 patterns), set A (1000 patterns), and set B (1000 patterns). This method evaluated performance (the number of attachment errors) on a  pattern set (validation set) after each complete pass through the training data (epoch). Series of three runs were performed that systematically varied the random starting weights. In each run the networks were trained for 40 epochs. In each run the weights of the epoch having the smallest error with respect to the validation set were stored. The weights corresponding to the best result obtained on the validation test in the three runs were selected and used to evaluate the performance in the test set. First, we used set A as validation set and set B as test, and afterwards we used set B as validation and set A as test. This experiment was replicated with two new partitions of the pattern set: two new training sets (18630 patterns) and 4 new validation/test sets of 1000 patterns each.</Paragraph>
    <Paragraph position="3"> Results showed in table 3 are the average accuracy over the six test sets (1000 patterns each) used. We performed three series of runs that varied the input encoding. In all these encodings, three tree cut thresholds were used: 10~o, 6~ and 2~o. The number of semantic classes in the input encoding ranged from 139 (10% cut) to 475 (2%) In the first encoding, the 4-tuple without extra information was used. The results for this case are shown in the 4-tuple column entry of table 3. In the second encoding, we added the prepositions the verbs select for their internal arguments, since English verbs with semantic similarity could select different prepositions (for example, accuse and blame). Verbs can be classified on the basis of the kind of prepositions they select. Adding this classification to the WordNet I classes in the input encoding improved the results (4-tuple + column entry of table 3).</Paragraph>
    <Paragraph position="4"> The 2% cut results were significantly better (p &lt; 0.02) than those of the 6% cut for 4-tuple and 4-tuple + encodings. Also, the results for the 4-tuple + condition were significanly better (p &lt; 0.01).</Paragraph>
    <Paragraph position="5"> For all simulations the momentum was 0.8, initial weight range 0.1. No exhaustive parameter exploration was carried out, so the results can still be improved.</Paragraph>
    <Paragraph position="6"> Some of the errors committed by the network can be attributed to an inadequate class assignment by WordNet. For instance, names of countries have only one sense, that of location. This sense is not appropriate in sentences like: Italy increased its sales to Spain; locations do not sell or buy anything, and the correct sense is social_group. Other mistakes come from what are known as reporting and aspectual verbs. For example in expressions like reported injuries to employees or iniliated lalks with the Soviets the nl has an argumental structure, and it is the element that imposes selectional restrictions on the PP. There is no good classification for these kinds of verbs in WordNet. Finally, collocations or idioms, which are very frequent, (e.g. lake a look, pay atlention), are not considered lexical units in the WSJ corpus. Their idiosyncratic behaviour introduces noise in the selectional restrictions acquisition process. Word-based models offer a clear advantage over class-based methods in these cases.</Paragraph>
  </Section>
class="xml-element"></Paper>