<?xml version="1.0" standalone="yes"?>
<Paper uid="C90-2045">
  <Title>An Interactive Japanese Parser for Machine Translation</Title>
  <Section position="6" start_page="0" end_page="0" type="evalu">
    <SectionTitle>
4 Evaluation
</SectionTitle>
    <Paragraph position="0"> One of tile claims of JAWB is that it can be used by non-expert users. To validate the claim, we conducted a comparative test with an expert user and a non-expert user. Figure 5 shows the results of the test. Subject A is one of the authors who actually developed the grammar. Subject B is a Japanese native speaker with no background in linguistics or computer science. Given an initial screen of dependency analysis, subject A spent 12.9 seconds on the average before making a correct parse tree.</Paragraph>
    <Paragraph position="1"> This period includes the time spent specifying the proper modifiees (1.1 times oii average) and verifying the system proposals, but does not include overheads such as the time spent choosing a new sentence to be analyzed and waiting for the system to look up dictionaries from a disk. The same task took 18.8 seconds for subject B. The important point here is that although the performance is somewhat different, tile parse trees generated by both subjects were essentially identical, a This means that, with a non-expert human user's help, JAWB is capable of producing very reliable parse trees fairly efficiently, although the efficiency can be increased by about 50% if an expert user uses it.</Paragraph>
    <Paragraph position="2"> Another yardstick for evaluating the system is the accuracy of the initial proposals. From 1,089 test sentences taken from actual newspaper arti~ 3There were differences when the sentence was truly ambiguous, in which case even a human user could not resolve the ambiguity without the context knowledge.</Paragraph>
    <Paragraph position="3"> cles, JAWB generated correct initial proposals for 507 sentences (47%), which means that, if it is used in a flfll-automatic mode, its accuracy is 47%. On the other hand, the system rejected two sentences as ungrammatical, which means that for 99.8% of the test sentences, JAWB was capable of producing correct parse trees with appropriate user interaction. null</Paragraph>
  </Section>
class="xml-element"></Paper>