<?xml version="1.0" standalone="yes"?>
<Paper uid="A94-1016">
  <Title>Three Heads are Better than One</Title>
  <Section position="4" start_page="98" end_page="98" type="metho">
    <SectionTitle>
3 TRANSLATION DELIVERY
SYSTEM
</SectionTitle>
    <Paragraph position="0"> Results of multi-engine MT were fed in our experiment into a translator's workstation (TWS) (Cohen et al., 1993), through which a translator either approved the system's output or modified it.</Paragraph>
    <Paragraph position="1"> The main option for human interaction in TWS currently is the Component Machine-Aided Translation (CMAT) editor (Frederking et hi., 1993a). The user sees the original source language text in one editor window, and phrases marked by double angle brackets in another, each of which is the first translation from a candidate chosen by the chart walk. Menus, function keys and mouse clicks are used to perform both regular and enhanced editing actions.</Paragraph>
    <Paragraph position="2"> The most important enhancement provided is the ability to select an alternate translation with a popup menu, and instantly replace the system's initially chosen candidate translation string, which becomes the first alternative in this menu if it is used again. The alternate translations are the other translations from the chosen component 3.</Paragraph>
    <Paragraph position="3"> As mentioned above, Figure 2 shows the sets of candidates in the best chart walk that are presented as choices to the human user through the CMAT editor in our example.</Paragraph>
  </Section>
  <Section position="5" start_page="98" end_page="99" type="metho">
    <SectionTitle>
TESTING AND EVALUATING
MULTI-ENGINE
PERFORMANCE
</SectionTitle>
    <Paragraph position="0"> Automatically assessing the utility of the multi-engine system relative to the engines taken separately would be a useful development tool. The best method we could find was counting the number of keystrokes in the TWS to convert the outputs of individual engines and the multi-engine configuration to a &amp;quot;canonical&amp;quot; human translation. A sample test on a passage of 2060 characters from the June 1993 evaluation of Pangloss is shown in figure 6.</Paragraph>
    <Paragraph position="1"> The difference in keystrokes was calculated as follows: one keystroke for deleting a character; two 3The CMAT editor may also include translations from other candidates, lower in the menu, if they have the same boundaries as the chosen candidate and the menu is not too long.</Paragraph>
    <Paragraph position="2">  and target languages. It is a weaker approach, but should go some distance in selecting between otherwise indistinguishable outputs.</Paragraph>
    <Paragraph position="3"> Another possible direction for future development would be to employ ideas from the area of heuristic search, and only run the highest-quality-score engine on each unit of source text. This assumes that we can reliably estimate scores in advance (not currently true for the expensive engines), and that the engines can be run on fragments. A less ambitious version of this idea would be to run the low-scoring engines only where there are gaps in the normally high-scoring engines.</Paragraph>
    <Paragraph position="4"> keystrokes for inserting a character; three keystrokes for deleting a word (in an editor with mouse action); three keystrokes plus the number of characters in the word being inserted for inserting a word. It is clear from the above table that the multi-engine configuration works better than any of our available individual engines, though it still does not reach the quality of a Level 2 translator.</Paragraph>
    <Paragraph position="5"> It is also clear that using keystrokes as a measure is not very satisfactory. It would be much better to make the comparison against the closest member of a set of equivalent paraphrastic translations, since there are many &amp;quot;correct&amp;quot; ways of translating a given input. However, this is predicated on the availability of a &amp;quot;paraphraser&amp;quot; system, developing which is not a trivial task.</Paragraph>
  </Section>
class="xml-element"></Paper>