<?xml version="1.0" standalone="yes"?>
<Paper uid="W06-2205">
  <Title>Recognition of synonyms by a lexical graph</Title>
  <Section position="5" start_page="33" end_page="35" type="metho">
    <SectionTitle>
4 Identification of synonyms
</SectionTitle>
    <Paragraph position="0"> The lexical graph is conceived as an instrument to identify semantic relations such as synonymy and hypernymy between lexical items represented by its vertices. The main  focus of our research was finding synonyms albeit some results can be immediately transferred for identification of hyponyms. To provide a quantitative measure of synonymy different similarity metrics were defined on the lexical graph. Given a word, the system uses the metric to calculate the closest vertices to the vertex that represents this word. The result is a ranked list of words sorted by the degree of synonymy in descending order. Every metric sim is normalized to be a probability measure so that given a vertex vi the value sim(vi,vj) can be interpreted as the probability of vj being synonym to vi. The normalization is performed for each metric sim by the following functions:</Paragraph>
    <Paragraph position="2"> for metrics that indicate maximum similarity to a vertex vi by a minimum value and</Paragraph>
    <Paragraph position="4"> for metrics that indicate maximum similarity to a vertex vi by a maximum value, where v1 ...vn are the set of graph vertices. In both cases the top-ranked word has the maximum likelihood of 1 to be a synonym of vi. The normalized ranked lists are used for the comparison of different metrics and the evaluation of the approach (see sec. 5).</Paragraph>
    <Paragraph position="5"> A similarity metric is supposed to assess the semantic similarity between two vertices of the lexical graph. Since the distance metric DistanceM used for calculation of distances between the vertices in the graph indicates how semantically related two vertices are, it can be used as a similarity metric. As the graph is directed, the distance metric is asymmetric, i.e. the distance from vj to vi does not have to be equal to the distance from vi to vj. The major drawback of the DistanceM is that it takes into account only one path between the examined vertices. Even though the shortest path indicates a strong semantic relation between the vertices, it is not sufficient to conclude synonymy that presupposes similar word senses.</Paragraph>
    <Paragraph position="6"> Therefore more evidence for strong semantic relation with the particular aspect of similar word senses should be incorporated in the similarity metric. The property neighbors of a vertex vi (adjacent vertices connected with vi by the property edge) play significant role in characterizing similar senses. If two terms share many characteristic properties, there is a strong evidence of their synonymy.</Paragraph>
    <Paragraph position="7"> A shared property can be regarded as a witness of the similarity of two word senses. There are other potential witnesses, e.g. transitive verbs shared by their direct objects; however, we restricted this investigation to the property neighbors as the most reliable witnesses.</Paragraph>
    <Paragraph position="8"> The simple method to incorporate the concept of the witnesses into the metric is to determine the number of common property neighbors:</Paragraph>
    <Paragraph position="10"> This method disregards, however, the different degree of correlation between the vertices and their property neighbors that is reflected by the length of property edges. A property is the more significant, the stronger the correlation between the property and the vertex is, that is the shorter the property edge is. The degree of synonymy of two terms depends therefore on the number of common properties and the lengths of paths between these terms leading through the properties. Analogously to the electric circuit one can see the single paths through different shared properties as channels in a parallel connection and path lengths as &amp;quot;synonymy resistances&amp;quot;. Since a bigger number of channels and smaller single resistances contribute to the decreasing of the total resistance (i.e. the evidence of synonymy increases), the idea of WeiPropM metric is to determine the similarity value analogously to the total resistance in a parallel connection:</Paragraph>
    <Paragraph position="12"> is the length of the path from vi to vj through pk and pk [?] prop(vi)[?]prop(vj).</Paragraph>
    <Paragraph position="13"> Another useful observation is that some properties are more valuable witnesses than the others. There are very general properties that are shared by many different terms and  some properties that are characteristic only for certain word senses. Thus the number of property neighbors of a property can be regarded as a measure of its quality (in the sense of characterizing the specific word meaning). WeiPropM integrates the quality of a prop-erty by weighting the paths leading through it by the number of its property neighbors:</Paragraph>
    <Paragraph position="15"> where pk [?] prop(vi)[?]prop(vj).</Paragraph>
    <Paragraph position="16"> WeiPropM measures the correlation between two terms based on the path lengths. Frequently occurring words tend to be ranked higher because the property edge lengths indirectly depend on the absolute word frequency. Because of high absolute frequency of words the frequency of their co-occurrence with different properties is generally also higher and the property edges are shorter. Therefore to compensate this deficiency (i.e. to eliminate the bias discussed in (Weeds et al., 2004)) an edge length from a property to a ranked term e(pk,vj) is weighted by the square root of its absolute frequency radicalBig freq(vj). Using the weighted edge length between the property and the ranked term we cannot any longer calculate the path length between vi and vj as the sum length(vi,pk,vj) = e(vi,pk) + e(pk,vj) [?]radicalBig freq(vj) because the multiplied second component significantly outweighs the first summand. Relative path length can be used instead where both components are adequately taken into account and added relatively to the minimum of the respective component: let min1 be min(e(vi,pa),...,e(vi,pn)) where pk [?] prop(vi) and min2 =</Paragraph>
    <Paragraph position="18"> searching for synonyms of vi the connection between vi and the property is more significant than the second component of the path the connection between the property and the ranked term vj. Therefore when calculating the relative path length the first component has to be weighted stronger (the examined ratio was 2:1). The corresponding metric can be defined as follows:</Paragraph>
    <Paragraph position="20"> As opposed to NaivePropM and WeiPropM FirstCompM is not symmetric because of the emphasis on the first component.</Paragraph>
  </Section>
  <Section position="6" start_page="35" end_page="200" type="metho">
    <SectionTitle>
5 Experiments
</SectionTitle>
    <Paragraph position="0"> For evaluation purposes a test corpus of 200 synonym sets was prepared consulting (OpenThesaurus, 2005). The corpus consists of 75 everyday words (e.g. &amp;quot;Pr&amp;quot;asident&amp;quot; (president), &amp;quot;Eingang&amp;quot; (entrance) &amp;quot;Gruppe&amp;quot; (group)), 60 abstract terms (e.g. &amp;quot;Ursache&amp;quot; (reason), &amp;quot;Element&amp;quot;, &amp;quot;Merkmal&amp;quot; (feature)) and 65 domain-specific words (e.g. &amp;quot;Software&amp;quot;, &amp;quot;Prozessor&amp;quot; (CPU)). The evaluation strategy is similar to that pursued in(Curran and Moens, 2002). The similarity metrics do not distinguish between different word senses returning synonyms of all senses of the polysemous words in a single ranked list. Therefore the synonym set of a word in the test corpus is the union of synonym sets of its senses.</Paragraph>
    <Paragraph position="1"> To provide a measure for overall performance and to compare the different metrics a function measuring the similarity score (SimS) was defined that assigns a score to a metric for correctly found synonyms among the 25 topranked. The function assigns 25 points to the correctly found top-ranked synonym of vi (SimS(0,vi) = 25) and 1 point to the synonym with the 25th rank (SimS(25,vi) = 1).</Paragraph>
    <Paragraph position="2"> The rank of a synonym is decreased only by false positives that are ranked higher (i.e. each of correctly identified top n synonyms has rank 0). In order to reward the top-ranked synonyms stronger the scoring function features a hyperbolic descent. For a synonym of vi with the rank x:</Paragraph>
    <Paragraph position="4"> To compare performance of different metrics the SimS values of the top 25 words in the ranked list were summed for each word of a test corpus. The total score of a similarity metric Sim issummationtext</Paragraph>
    <Paragraph position="6"> where RankedList(vi,j) returns the word at the position j from the ranked list produced by Sim for vi and v1,...,v200 are the words of the test corpus.</Paragraph>
    <Paragraph position="7"> Besides, a combined precision and recall measure P was used to evaluate the ranked lists. Given the word vi, we examined the first n words (n = 1,5,25,100) of the ranked list returned by a similarity metric for vi whether they belong to the synset(vi) of the test corpus. P(n) will measure precision if n is less than the size of the synset(vi) because the maximum recall can not be reached for such n and recall otherwise because maximum precision cannot be reached for n &gt; |synset(vi)|. The P values were averaged over 200 words.</Paragraph>
    <Paragraph position="8"> Table 1 presents the result of evaluating the similarity metrics introduced in sec. 4. The results of DistanceM confirm that regarding distance between two vertices alone is not sufficient to conclude their synonymy. DistanceM finds many related terms ranking general words with many outgoing and incoming edges higher, but it lacks the features providing the particular evidence of synonymy.</Paragraph>
    <Paragraph position="9"> NaivePropM is clearly outperformed by the both weighted metrics. The improvement relative to the DistanceM and acceptable precision of the top-ranked synonyms P(1) show that considering shared properties is an adequate approach to recognition of synonyms.</Paragraph>
    <Paragraph position="10"> Ignoring the strength of semantic relation indicated by the graph and the quality of properties is the reason for the big gap in the total score and recall value (P(100)). Both weighted metrics achieved results comparable with those reported by Curran and Moens in (Curran and Moens, 2002) and Turney in (Turney, 2001). Best results of FirstCompM confirm that the criteria identified in sec. 4 such as generality of a property, abstraction from the absolute word frequency etc. are relevant for identification of synonyms. FirstCompM performed particularly better in finding synonyms with the low frequency of occurrence. null In another set of experiments we investigated the influence of the size of the text corpus (cf. fig. 3). The plausible assumption is the more texts are processed, the better the semantic connections between terms are reflected by the graph, the more promising results are expected. The fact that the number of vertices does not grow proportionally to the size of text corpus can be explained by word recurrence and growing filtering threshold th. However, the number of edges increases linearly and reflects the improving semantic coverage. As expected, every metric performs considerably better on bigger graphs. While NaivePropM seems to converge after three volumes, the both weighted metrics behave strictly monotonically increasing. Hence an improvement of results can be expected on bigger corpora. On the small text corpora the results of single metrics do not differ significantly since there is not sufficient semantic information captured by the graph, i.e. the edge and path lengths do not fully reflect the semantic relations between the words. The scores of both weighted metrics grow, though, much faster than that of NaivePropM. FirstCompM achieves the highest gradient demonstrating the biggest potential of leveraging the growing graph for finding synonymy.</Paragraph>
    <Paragraph position="11">  To examine the influence of the word categories results on the subsets of the text corpus corresponding to a category are compared. All metrics show similar behavior, therefore we restrict the analysis to the P values of  FirstCompM (fig. 4). Synonyms of domain-specific words are recognized better than those of abstract and everyday words. Their semantics are better reflected by the technically oriented texts. The P values for abstract and everyday words are pretty similar except for the high precision of top-ranked abstract synonyms. Everyday words suffer from the fact that their properties are often too general to uniquely characterize them, which involves loss of precision. Abstract words can be extremely polysemous and have many subtle aspects that are not sufficiently covered by the texts of computer journals.</Paragraph>
    <Paragraph position="12">  gory (results of FirstCompM metric) To test whether the metrics perform better for the more frequent words the test set was divided in 9 disjunctive frequency clusters (table 2). FirstCompM achieved considerably better results for very frequently occurring words ([?] 4000 occurrences). This confirms indirectly the better results on the bigger text corpora: while low frequency does not exclude random influence, frequent occurrence involves adequate capturing of the word semantics in the graph by inserting and adjusting all relevant property edges. These results do not contradict the conclusion that FirstCompM is not biased towards words with a certain frequency because the mentioned bias pertains to retrieval of synonyms with a certain frequency, whereas in this experiment the performance for different word frequencies of queried words is compared.</Paragraph>
  </Section>
class="xml-element"></Paper>