<?xml version="1.0" standalone="yes"?>
<Paper uid="W06-3107">
  <Title>Searching for alignments in SMT. A novel approach based on an Estimation of Distribution Algorithm [?]</Title>
  <Section position="7" start_page="51" end_page="51" type="evalu">
    <SectionTitle>
5.2 Results
</SectionTitle>
    <Paragraph position="0"> In Tables 2, 3 and 4 the results obtained from the different tasks are presented. The results achieved by the technique proposed in this paper are compared with the best results presented in the shared tasks described in (Mihalcea and Pedersen, 2003) (Joel Martin, 2005). The results obtained by the GIZA++ hill-climbing algorithm are also presented.</Paragraph>
    <Paragraph position="1"> In these tables, the mean and the variance of the results obtained in ten executions of the search algorithm are shown. According to the small variances observed in the results we can conclude that the non-deterministic nature of this approach it is not statistically significant.</Paragraph>
    <Paragraph position="2"> According to these results, the proposed EDA-based search is very competitive with respect to the best result presented in the two shared task.</Paragraph>
    <Paragraph position="3"> In addition to these results, additional experiments were carried out in to evaluate the actual behavior of the search algorithm. These experiments were focused on measuring the quality of the algorithm, distinguishing between the errors produced by the search process itself and the errors produced by the model that leads the search (i.e, the errors introduced by the fitness function). To this end, the next approach was adopted. Firstly, the (bidirectional) reference alignments used in the computation of the Alignment Error Rate were split into two sets of unidirectional alignments. Owing to the fact that there is no exact method to perform this decomposition, we employed the method described in the following way. For each reference alignment, all the possible decompositions into unidirectional alignments were perfomed, scoring each of them with the evaluation function F(a) = p(f,a|e) defined in section (3), and being selected the best one, aref.</Paragraph>
    <Paragraph position="4"> Afterwards, this alignment was compared with the solution provided by the EDA, aeda . This comparison was made for each sentence in the test set, being measuried the AER for both alignments as well as the value of the fitness function. At this point, we can say that a model-error is produced if F(aeda) &gt; F(aref). In addition, we can say that a search-error is produced if F(aeda) &lt; F(aref). In table 5, a summary for both kinds of errors for the English-Romanian 2005 task is shown. In this table we can also see that these results correlate with the AER figures.</Paragraph>
    <Paragraph position="5"> These experiments show that most of the errors were not due to the search process itself but to another different factors. From this, we can conclude that, on the one hand, the model used to lead the search should be improved and, on the other, different techniques for symmetrization should be explored. null</Paragraph>
  </Section>
class="xml-element"></Paper>