<?xml version="1.0" standalone="yes"?>
<Paper uid="A92-1008">
  <Title>Generating Spatial Descriptions for Cross-modal References</Title>
  <Section position="4" start_page="57" end_page="60" type="metho">
    <SectionTitle>
3 Basic Localisation Procedures
</SectionTitle>
    <Paragraph position="0"> In this section we present matrix-oriented localisation procedures for absolute and relative localisations. As mentioned in section 2.2, both the horizontal and vertical relation of the primary object are given in case of a composite localisation. This suggests that composite localisations are composed of elementary localisations.</Paragraph>
    <Paragraph position="1"> The procedures presented here, though, behave differently: for the sake of efficiency they compute the composite localisations first and derive the elementary localisations from these previously computed localisation results.</Paragraph>
    <Section position="1" start_page="58" end_page="59" type="sub_section">
      <SectionTitle>
3.1 Absolute loealisations
</SectionTitle>
      <Paragraph position="0"> We approximate the center of the picture with a rectangle whose horizontal and vertical extension is one third of the horizontal and vertical extension of the picture.</Paragraph>
      <Paragraph position="1"> Figure 8 shows the construction of the horizontal and vertical reference system according to the rectangular center region.</Paragraph>
      <Paragraph position="2"> vertical right lell center</Paragraph>
      <Paragraph position="4"> reference system Before describing the evaluation function for cornposite localisations, we give a few definitions: * The horizontal reference system is abbreviated by</Paragraph>
      <Paragraph position="6"> sations are denoted by CLOC = XLOC xYLOC. Both reference systems together are described with ULOC = XLOCI.JYLOC.</Paragraph>
      <Paragraph position="7"> * The constant CENTER denotes the center rectangle of a given picture.</Paragraph>
      <Paragraph position="8"> * POLY denotes the set of all polygons that can appear in a picture. For given polygons P1 and P2 tile associative and commutative operator N,  (&amp;quot;1 : POLY X POLY ~ POLY computes the intersection polygon. The empty polygon is denoted by P0. The following holds: VP E POLY : P$ 71 P = pnD~ = P~.</Paragraph>
      <Paragraph position="9"> * The fimction PR (Partial Rectangle), PR : CLOC x POLY ~ POLY, computes the rectangle correspond null ing to a given composite localisation and the rectangle partition of the picture induced by a given polygon. For example PR((left,top), (:ENTER) computes the upper left rectangle according to the partition scheme shown in figure 8.</Paragraph>
      <Paragraph position="10"> * !R denotes the set of the real numbers. Given a polygon P, the fimction f, f : POLY ~ N computes the area of a polygon. It is f(P~) = O.</Paragraph>
      <Paragraph position="11"> The applicability degree of a composite localisation evaluates how good the position of the object in question is described by that particular localisation. We define the applicability degree as the portion of the area of the object that lies in the rectangle of the picture that corresponds to the composite localisation and the rectangle partition of that picture. Thus we can define A~ as follows:</Paragraph>
      <Paragraph position="13"> definition: the applicability degree Ae of an elementary localisation is determined by the portion of the area of the object that lies in the corresponding row or column of the picture. As already mentioned at the beginning of this section we can write A~ in terms of A~ :</Paragraph>
      <Paragraph position="15"> A~ and A~ compute tile applicability for the horizontal and vertical dimension by summing up the applicability degrees of the corresponding composite localisations.</Paragraph>
      <Paragraph position="16"> They are combined ill AC/ order to have a function that is defined oll both dimensions, i.e., ULOC.</Paragraph>
      <Paragraph position="17"> With respect to figure 9 we get:</Paragraph>
      <Paragraph position="19"> As argued in l)aragraph 2.4 corner localisations are similar to composite ('left'/'right', 'top'/'bottorn')localisations, but less general. This property can be modelled by corner regions that are smaller than tim corner regions for absolute localisations. In turn, these corner regions correspond to a larger center as shown in figure 10. Thus we can compute corner localisations just by changing the size of the center.</Paragraph>
      <Paragraph position="21"> Instead of 1/3 as for absolute localisations we take 4/5 of the horizontal and vertical extension of the picture for the extended center.</Paragraph>
    </Section>
    <Section position="2" start_page="59" end_page="60" type="sub_section">
      <SectionTitle>
3.2 Relative localisations
</SectionTitle>
      <Paragraph position="0"> The localisation procedure for relative localisations is similar to the one for absolute localisations. One major difference is that now the construction of the horizonta.l and vertical reference frame is done with respect to a given reference object and not to the implicit assumed center of the picture (c.f. figure 11). The second difference concerns the computation of the applicability degree: for relative localisations, not only the portion of an area is taken into account, but also the distance between the primary object and the reference object.</Paragraph>
      <Paragraph position="1"> vertical left center right  The basic idea for the evaluation of the distance between primary object and reference object is adopted from the C,1TYTOUR system: first we compute the center of gravity for the primary object. Then we determine its coordinates with respect to the reference system established by the reference object. Finally we use these coordinates for the computation of the applicability degree. Figure 12 illustrates the various factors that affect  the applicability of an 'above'-localisation: 1. The applicability degree decreases with an increasing vertical distance. In Part A of figure 12 the ap2. null</Paragraph>
      <Paragraph position="3"> plicability degree for &amp;quot;P1 is above REFO&amp;quot; is greater than for &amp;quot;P2 is above REFO.&amp;quot; The applicability degree decreases with an increasing horizontal distance. In Part B the applicability degree for &amp;quot;P3 is above REFO&amp;quot; is greater than for &amp;quot;P4 is above REFO.&amp;quot; If the horizontal and vertical distances increase by the same amount, then the applicability degree decreases more with the increasing horizontal distance than with the increasing vertical distance. This is shown in Part C: the applicability degree for &amp;quot;P6 is above REFO&amp;quot; is greater than for &amp;quot;P7 is above REFO&amp;quot;, although the vertical distance between P5 and P6 and the horizontal distance between P5 and</Paragraph>
      <Paragraph position="5"> Let eval denote the function that evaluates the distance between a point and a rectangle according to the criteria mentioned above. Let further POINT denote the set of all points within a picture and RECT C POLY the set of all rectangular polygons. Then the signature oi eval can be written as2: eval : CLOC x POINT x RECT ~ ~}~ Now we are almost able to define the function Ac, which computes the applicability degree of a composite localisation. Let CG, CG : POLY ~ POINT , compute the center of gravity for a polygon and let further SR, SR : POLY ~ RECT , compute the smallest surrounding rectangle for a polygon. Then the applicability degree Ac of a composite localisation can be defined as:</Paragraph>
      <Paragraph position="7"> 2In reality eval is slightly more complicated because it maps into ~' x ~' and not only into 3. The reason for this is that the different evaluation of increasing vertical and horizontal distances can result in different evaluations for points to which both a horizontal or vertical localisation can be applied. E.g., P7 in figure 12 would get a different evaluation foJ an 'above'- than for a 'right of'-localisation. Therefore, thes~ two values would be grouped to a tuple. For the computatior of an elementary localisation 1 E XLOC we would sum up th~ first component of the tuple. If 1 E YLOC, we take the seconc component. We abstract from this detail in order to mak~ the principle of the procedure clearer.</Paragraph>
      <Paragraph position="8">  p is tile part of the primary object that lies in the rectangle corresponding to the composite localisation I.</Paragraph>
      <Paragraph position="9"> The factor w weighs the result of eval according to the portion of the area of the primary object that lies in the rectangle corresponding to I.</Paragraph>
      <Paragraph position="10"> Now the definition of Ae, the applicability degree for an elementary localisation, can be given in terms of A~ again:</Paragraph>
      <Paragraph position="12"> Ae(I, LO, REFO) = I A~(I, LO, REFO) if l E XLOC * A~(l, LO, REFO) if l E YLOC This means that the applicability degree Ae for a primary object LO is the sum of the coml)osite localisations for tlle corresponding row or colunm of tile reference fr anle.</Paragraph>
      <Paragraph position="13"> For figure 1:3 we get, the following results: A~((x-center, top), LO, REFO) -5- l eval((x-center, top), P1, SR(REFO)</Paragraph>
    </Section>
  </Section>
  <Section position="5" start_page="60" end_page="61" type="metho">
    <SectionTitle>
4 A generic localisation procedure for
</SectionTitle>
    <Paragraph position="0"> absolute and relative localisations The similarities between the localisation procedures discussed in the previous section allow us to design one generic localisation procedure that can be specialised to a procedure for absolute, relative or corner localisations. Given the primary object, LO and the reference object REFO the first step is to determine the 3 x 3 matrix M n, which contains the intersection polygons of LO and the partial rectangles in the picture with respect to REFO.</Paragraph>
    <Paragraph position="1"> For relative localisations, REFO varies, for absolute localisations and corner localisations the parameter is set to either the normal or the extended center area (c.f.</Paragraph>
    <Paragraph position="2"> section 3.1). Thus, for x E XLOC, y E YLOC we compute</Paragraph>
    <Paragraph position="4"> The second step is the computation of the evaluation matrix M A, which contains the applicability degrees of the composite localisations. The computation requires a fimction E, E : POLY xPOLY xPOLY ~ ~. E corresponds exactly to the flmction Ae for absolute and relative localisations in section 3.1 and 3.2. The only difference results from tile previous computation of Mn: tile subexpression p = PR((x, y), REFO) M LO is factored from AC/ and therefore computed only once.</Paragraph>
    <Paragraph position="5"> MAu = E(M2,u, LO, REFO) The third step is the computation of the elementary localisations. The vector )~ contains the evaluations of the horizontal localisations and \]7 the ones for the vertical localisations: yEYLOC xEXLOC This means that we have -~t = Ae(l) for l E XLOC and = A~(l) for l E YLOC.</Paragraph>
    <Paragraph position="6"> Finally, we can determine the best composite and elementary localisation and their applicability degrees by computing the maximum value of M A and X or )7 respectively. null For figure 13 we get</Paragraph>
    <Paragraph position="8"> ite localisation is &amp;quot;(right, top)&amp;quot; with applicability degree 0.4:3. The best elementary localisation is &amp;quot;top&amp;quot; with applicability degree 0.66.</Paragraph>
    <Paragraph position="9"> 5 Localising objects in a complex scene In the previous sections we considered pictures with a minimal number of objects. In order to deal with more complex object configurations the localisation procedures presented above have to be extended. Tile new task is no longer &amp;quot;Localise LO with respect to REFO!'&amp;quot; but &amp;quot;Given a set of REFO candidates, choose tile best one for LO!&amp;quot;  In order to reduce the search space for REFO candidates, frst a kind of 'between'-test is applied to the set of possible reference objects. The idea behind this test is that an exclusion procedure based on simple geometric overlapping tests can be performed more efficiently than a comparison of applicability degrees that have to be computed by the rather complex localisation procedures. An example is given in figure 14: When searching for a suitable reference object for object A in figure 14, object D would be ruled out because object B is found in the 'between'-area of A and D. /Jiiiiiiiiiiiiiiiiiiiiiiiiiii &amp;quot; l</Paragraph>
    <Paragraph position="11"> The deterinination of the best reference object raises the problem of ambiguity. Not only is the applicability degree of a localisation important, but also whether the use of the reference object would result in an ambiguous localisation. In that case, a different reference object has to be chosen. If all possible localisations are ambiguous, then the particular object cannot be loealised at. all. E.g., in Part A of figure 15 object D could be localised as being either &amp;quot;above A&amp;quot; or &amp;quot;to the right of (:.&amp;quot; But the first localisation is ambiguous because both, C and D, are &amp;quot;above A.&amp;quot; A. B. C.</Paragraph>
    <Paragraph position="12"> Figure 15: Ambiguous reference objects With respect to elementary and composite localisations we distinguish three cases of ambiguity:</Paragraph>
  </Section>
  <Section position="6" start_page="61" end_page="61" type="metho">
    <SectionTitle>
1. In Part A of figure 15, the localisation of object (I or
</SectionTitle>
    <Paragraph position="0"> D would be ambiguous with respect to A because for both objects the composite localisations, (x-center, top), are equal.</Paragraph>
  </Section>
  <Section position="7" start_page="61" end_page="61" type="metho">
    <SectionTitle>
2. In Part B a composite localisation cannot be applied
</SectionTitle>
    <Paragraph position="0"> to object D (neither &amp;quot;D is above and to the right of A&amp;quot; nor &amp;quot;D is immediately above A&amp;quot; are adequate) and its elementary localisation, 'top', is part of the composite localisation, (x-center, top), of object C.</Paragraph>
  </Section>
  <Section position="8" start_page="61" end_page="61" type="metho">
    <SectionTitle>
3. In Part C a composite localisation can be applied
</SectionTitle>
    <Paragraph position="0"> neither to C nor to D and their elementary localisations, 'top', are equal.</Paragraph>
  </Section>
  <Section position="9" start_page="61" end_page="61" type="metho">
    <SectionTitle>
6 Localising Groups of Objects
</SectionTitle>
    <Paragraph position="0"> Control knobs and switches are often grouped together in a control panel in order to provide for easier operation of technical devices. Moreover spatially adjacent objects can also be grouped as one perceptual unit according to the 'law of the good gestalt' in Gestalt psychology (\[Murch and Woodworth, 1978\]). Thus the possibility to generate loealisations with respect to a given group structure is neccessary for the &amp;quot;naturalness&amp;quot; of a localisation. Besides this, group localisations are also useful if the objects in the immediate neighbourhood of the primary object have exactly the same properties (c.f.</Paragraph>
    <Paragraph position="1"> \[Wahlster el al., 1978\]). In this case, the primary object can be localised with respect to its group and has not to be localised with respect to the whole scene, which could have resulted in an ambiguous localisation.</Paragraph>
    <Paragraph position="2"> For our localisation procedures this means that groups can function as a reference object as well as a primary object.. In addition, objects can be localised absolutely with respect to the group they are contained in. In figure 16 object B would be localised as the object &amp;quot;to the right of the triangles.&amp;quot; Vice versa we can say &amp;quot;The triangles to the left of object B&amp;quot; and we can localise object A as being &amp;quot;the upper left of the triangles that are to the left of B.&amp;quot; Figure 16: Group localisations The last example also illustrates the hierarchical character of group localisations: An object can be localisec absolutely within a group. This group might be localisec again within a surrounding group or -- if there is non( -- this group can be localised relatively with respect t( another (group of) object(s).</Paragraph>
    <Paragraph position="3"> The algorithm for group localisations cannot detecl group hierarchies. Instead it expects a tree representation of tile group hierarchy as an input. The output con sists of two parts: According to the depth of the grout tree the algorithm computes a chain of absolute locali sations. In addition the outermost surrounding group o the primary object is localised relatively to an optiona (group of) reference object(s).</Paragraph>
  </Section>
class="xml-element"></Paper>