<?xml version="1.0" standalone="yes"?>
<Paper uid="C04-1171">
  <Title>A System for Generating Descriptions of Sets of Objects in a Rich Variety</Title>
  <Section position="5" start_page="0" end_page="0" type="evalu">
    <SectionTitle>
4 Experimental Results
</SectionTitle>
    <Paragraph position="0"> We have implemented the algoritm in Common Lisp, on an Intel Pentium processor with 2600 MHz. In the following elaborations, we use natural language descriptions for reasons of readability, even though our algorithm only produces boolean combinations of descriptors.</Paragraph>
    <Paragraph position="1"> We evaluate our algorithm from three perspectives: 1) effects of the linguistically motivated restrictions, 2) effectiveness of the cut-off techniques, and 3) the behavior in scaling up for larger examples. For this purpose, we have built all subsets of two, three, and four vehicles, out of the vehicles x1 to x6, which yields 50 cases.</Paragraph>
    <Paragraph position="2"> In order to test the effects of the linguistically motivated reductions, we have used two versions  of the 50 cases, one with all properties, and one without size and age. In these runs, the maximum number of descriptors chosen was 5, and search trees grew up to 9 with and 20 nodes without using the linguistically motivated reductions. The average search times were 127.7 resp. 440.5 msec, with a maximum of 950 resp. 2590 msec.</Paragraph>
    <Paragraph position="3"> In order to compare the effectiveness of the cut-off techniques, we have run the same sample of 100 cases (50 with and 50 without size and age), with all combinations of at least one cut-off technique. Table 1 illustrates the results. Among others, they demonstrate that search times are not proportional to tree sizes, since a lot of effort is devoted to justify the avoidance of expansions, which varies among cut-off techniques. It turns out that the value cut-off is the most effective one, which underpins the importance of finding a solution quickly. Looking at individual examples reveals that the complementary effects of dominance and complexity cut-offs are significant only for examples with larger solutions. Finally, we have tested the algorithm's scalability, by increasing the number of distractors, with up to 25 vehicles (similar to x1 to x12, but distinct from one another). The same 100 cases have been used as before, with all cut-off criteria. The results appear in Table 2. They demonstrate that the problem tends to get unmanagable for more than 12 distractors in both search time and number of descriptors needed for identification, the latter being the reason for the former.</Paragraph>
    <Paragraph position="4"> However, descriptions consisting of up to 10 descriptors are unlikely to be understandable for humans, anyway - consider, for example, &amp;quot;the cars which are not blue, are old or stand in the center, are new or stand on the right side, are big or not white, and are small or not red&amp;quot; (108110 msec, identifying x3, x4, and x6 out of 25 vehicles). For such complicated cases, identifying objects is broken down into simpler tasks (see Section 3.2). Conversely, useful results may be obtained for a large number of distractors - for example, &amp;quot;the old cars on the right side&amp;quot; (120 msec, identifying x3 and x6 out of 25 vehicles).</Paragraph>
    <Paragraph position="5"> nr. of distractors</Paragraph>
  </Section>
class="xml-element"></Paper>