<?xml version="1.0" standalone="yes"?>
<Paper uid="J04-2003">
  <Title>c(c) 2004 Association for Computational Linguistics Statistical Machine Translation with Scarce Resources Using Morpho-syntactic Information</Title>
  <Section position="9" start_page="199" end_page="201" type="evalu">
    <SectionTitle>
7.5 Results for Log-Linear Lexicon Combination
</SectionTitle>
    <Paragraph position="0"> 7.5.1 Results on the Verbmobil Task. As was pointed out in Section 4, the hierarchical lexicon is expected to be especially useful in cases in which many of the inflected word forms to be accounted for in test do not occur during training. To systematically investigate the model's generalization capability, it has been applied on the three different setups described in Section 7.4. The training procedure was the one proposed in Section 6, which includes restructuring transformations in training and test. Table 9 summarizes the improvement achieved for all three setups.</Paragraph>
    <Paragraph position="1"> Training on 58,000 sentences plus conventional dictionary: Compared to the effect of restructuring, the additional improvement achieved with the hierarchical lexicon is relatively small in this setup. The combination of all methods results in a relative improvement in terms of SSER of almost 13% and in terms of information ISER of more than 16% as compared to the baseline.</Paragraph>
    <Paragraph position="2"> Training on 5,000 sentences plus conventional dictionary: Restructuring alone can improve the translation quality from 37.3% to 33.6%. The benefit from the hierarchical lexicon is larger in this setup, and the resulting in SSER is 31.8%. This is a relative improvement of almost 15%. The relative improvement in terms of ISER is almost 22%. Note that by applying the methods  Computational Linguistics Volume 30, Number 2 Table 9 Results for hierarchical lexicon models and translation with scarce resources. &amp;quot;Restructuring&amp;quot; entails treatment of question inversion and separated verb prefixes as well as merging of phrases in both languages. A conventional dictionary is available in all three setups. The language model is always trained on the full monolingual English corpus. Task: Verbmobil. Testing on 527 sentences (Test and Develop).</Paragraph>
    <Paragraph position="3"> Number of sentences for training BLEU m-WER SSER ISER  proposed here, the corpus for training can be reduced to less than 10% of the original size while increasing the SSER only from 30.2% to 31.8% compared to the baseline when using the full corpus.</Paragraph>
    <Paragraph position="4"> Training only on conventional dictionary: In this setup the impact of the hierarchical lexicon is clearly larger than the effect of the restructuring methods, because here the data sparseness problem is much more important than the word order problem. The overall relative reduction in terms of SSER is 13.7% and in terms of ISER 19.1%. An error rate of about 52% is still very poor, but it is close to what might be acceptable when only the gist of the translated document is needed, as is the case in the framework of document classification or multilingual information retrieval.</Paragraph>
    <Paragraph position="5"> Examples taken from the Verbmobil Eval-2000 test set are given in Table 10. Smoothing the lexicon probabilities over the inflected forms of the same lemma enables the translation of sind as would instead of are. The smoothed lexicon contains the translation convenient for any inflected form of bequem. The comparative more convenient would be the completely correct translation. The last two examples in the table demonstrate the effect of the disambiguating analyzer, which on the basis of the sentence context identifies Zimmer as plural (it has been translated into the singular form room by the baseline system) and das as an article to be translated by the instead of a pronoun which would be translated as that. The last example demonstrates that over-fitting on domain-specific training can be problematic in some cases: Generally, because is a good translation for the co-ordinating conjunction denn, but in the appointmentscheduling domain, denn is often an adverb, and it often occurs in the same sentence as dann,asinWie w&amp;quot;are es denn dann?. The translation for this sentence is something like How about then?. Because of the frequency of this domain-specific language use, the word form denn is often aligned to then in the training corpus. The hierarchical  Niessen and Ney SMT with Scarce Resources Table 10 Examples of the effect of the hierarchical lexicon.</Paragraph>
    <Paragraph position="6"> Input sind Sie mit einem Doppelzimmer einverstanden? Baseline are you agree with a double room? Hierarchical lexicon would you agree with a double room? Input mit dem Zug ist es bequemer.</Paragraph>
    <Paragraph position="7"> Baseline by train it is UNKNOWN-bequemer.</Paragraph>
    <Paragraph position="8"> Hierarchical lexicon by train it is convenient.</Paragraph>
    <Paragraph position="9"> Input wir haben zwei Zimmer.</Paragraph>
    <Paragraph position="10"> Baseline we have two room.</Paragraph>
    <Paragraph position="11"> Hierarchical lexicon we have two rooms.</Paragraph>
    <Paragraph position="12"> Input ich w &amp;quot;urde das Hilton vorschlagen denn es ist das beste.</Paragraph>
    <Paragraph position="13"> Baseline I would suggest that Hilton then it is the best.</Paragraph>
    <Paragraph position="14"> Hierarchical lexicon I would suggest the Hilton because it is the best.</Paragraph>
    <Paragraph position="15"> lexicon distinguishes the adverb reading and the conjunction reading, and the correct translation because is the highest-ranking one for the conjunction.</Paragraph>
    <Paragraph position="16">  corpus from the Nespole! project (see Section 7.1 for a description). From Table 5 it is obvious that this task is an example of very scarce training data, and it is thus interesting to test the performance of the methods proposed in this article on this task. The same conventional dictionary as was used for the experiments on Verbmobil data (cf. Table 6) complemented the small bilingual training corpus. Furthermore, the (monolingual) English part of the Verbmobil corpus was used in addition to the English part of the Nespole! corpus for training the language model. Table 11 summarizes the results. Information items have not been defined for this test set. An overall relative improvement of 16.5% in the SSER can be achieved.</Paragraph>
  </Section>
class="xml-element"></Paper>