<?xml version="1.0" standalone="yes"?>
<Paper uid="P95-1027">
  <Title>A Quantitative Evaluation of Linguistic Tests for the Automatic Prediction of Semantic Markedness</Title>
  <Section position="9" start_page="201" end_page="202" type="concl">
    <SectionTitle>
8 Conclusions and Future Work
</SectionTitle>
    <Paragraph position="0"> We have presented a quantitative analysis of the performance of measurable linguistic tests for the selection of the semantically unmarked term out of a pair of antonymous adjectives. The analysis shows that a simple test, word frequency, outperforms more complicated tests, and also dominates them in terms of information content. Some of the tests that have been proposed in the linguistics literature, notably tests that are based on the formal complexity and differentiation properties of the words; fail to give any useful information at all, at least with the approximations we used for them (Section 3). On the other hand, tests based on morphological productivity are valid, although not as accurate as frequency. Naturally, the validity of our results depends on the quality of our measurements. While for most of the variables our measurements are necessarily apsit should be noted here that the independence assumption of the sign test is mildly violated in these repeated runs, since the scores depend on collections of independent samples from a finite population. This mild dependence will increase somewhat the probabilities under the true null distribution, but we can be confident that probabilities such as 0.08% will remain significant.</Paragraph>
    <Paragraph position="1">  performance relative to the simple frequency test equal to or larger than the observed one is listed in the P- Value column for each complex predictor.</Paragraph>
    <Paragraph position="2"> proximate, we believe that they are nevertheless of acceptable accuracy since (1) we used a representative corpus; (2) we selected both a large sample of adjective pairs and a large number of frequent adjectives to avoid sparse data problems; (3) the procedure of identifying secondary words for indirect measurements based on morphological productivity operates with high recall and precision; and (4) the mapping of the linguistic tests to comparisons of quantitative variables was in most cases straightforward, and always at least plausible.</Paragraph>
    <Paragraph position="3"> The analysis of the linguistic tests and their combinations has also led to a computational method for the determination of semantic markedness. The method is completely automatic and produces accurate results at 82% of the cases. We consider this performance reasonably good, especially since no previous automatic method for the task has been proposed. While we used a fixed set of 449 adjectives for our analysis, the number of adjectives in unrestricted text is much higher, as we noted in Section 2. This multitude of adjectives, combined with the dependence of semantic markedness on the domain, makes the manual identification of markedness values impractical.</Paragraph>
    <Paragraph position="4"> In the future, we plan to expand our analysis to other classes of antonymous words, particularly verbs which are notoriously difficult to analyze semantically (Levin, 1993). A similar methodology can be applied to identify unmarked (positive) versus marked (negative) terms in pairs such as agree: dissent.</Paragraph>
  </Section>
class="xml-element"></Paper>