<?xml version="1.0" standalone="yes"?>
<Paper uid="T78-1023">
  <Title>ON THE ONTOLOGICAL STATUS OF VISUAL MENTAL IMAGES</Title>
  <Section position="1" start_page="0" end_page="168" type="abstr">
    <SectionTitle>
ON THE ONTOLOGICAL STATUS OF VISUAL MENTAL IMAGES
</SectionTitle>
    <Paragraph position="0"> There has long been considerable controversy over the ontological status of mental images. Most recently, members of the A.I. community have argued for the sufficiency of &amp;quot;propositional representation&amp;quot; and have resisted the notion that other sorts of representations are functional in the human mind. The purpose of this paper is to review what I take to be the best evidence that images are distinct functional representations in human memory. Before reviewing these data, however, I offer a preliminary definition of what I mean by a &amp;quot;visual mental image.&amp;quot; This definition arises out of the &amp;quot;cathode ray tube&amp;quot; metaphor originally introduced in Kosslyn (1974, 1975, 1976) and later implemented in a computer simulation by Kosslyn &amp; Shwartz (1977a, in press). On this view, images are spatial representations in active memory generated from more abstract representations in Long-term memory; these spatial representations are able to be interpreted (&amp;quot;inspected&amp;quot;) by procedures that classify them into various semantic categories.</Paragraph>
    <Paragraph position="1"> 1.0 A preliminar 7 definition of a visual mental image I wish to define a &amp;quot;visual mental image&amp;quot; in terms of five basic kinds of properties. Images are often distinguished from more discrete, propositional or linguistic representations because they supposedly have &amp;quot;analogue&amp;quot; properties. Thus, the first two properties noted below describe analogue representations as a class.</Paragraph>
    <Paragraph position="2"> Goodman (1968), Palmer (in press), Shepard (1975), Sloman (1975), and others have provided informative and detailed discussions of relevance here, and I will draw freely on these sources in the present discussion.</Paragraph>
    <Paragraph position="3"> 1) Images can capture continuous variations in shape. This continuity The work reported here was supported by NSF Grants BNS 76-16987 and BNS 77-21782. I wish to thank Willa Rouder for her assistanCean &amp;quot;abstract first-order isomorphism.&amp;quot; in preparing the manuscript.</Paragraph>
    <Paragraph position="4"> property implies that image representations are both semantically and syntactically &amp;quot;dense&amp;quot; or &amp;quot;undifferentiated&amp;quot; in the extreme (Goodman, 1968, p. 136 ff.). For example, a reading on a tire pressure gauge is an analogue representation to some extent, because every reading along the continuous scale has meaning (and so it is semantically dense); if the gauge had an infinity of markings of pounds-per-square inch, the scale would be syntactically dense and readings on it would be purely analogue. In contrast, discrete representations are not semantically or syntactically dense, but are differentiated (i.e., separable and distinct). For example, each reading of a digital clock, in contrast to the traditional dial variety, is entirely unambiguous in terms of its identity (i.edeg, is syntactically distinct) and its meaning (i.e., is semantically distinct).</Paragraph>
    <Paragraph position="5"> Images are both semantically and syntacically dense.</Paragraph>
    <Paragraph position="6"> 2) Part and parcel of the continuity property is the property that analogue representations are not arbitrarily related to their referents. Because analogue representations can be arranged on a continuum (e.g., of size), a symbol indicating a value falling between two others (e.g., an intermediate size) must refer to a value of the referent falling between the two indicated by the others (e.g., an object of intermediate size).</Paragraph>
    <Paragraph position="7"> Hence, unlike discrete representations, any given analogue representation cannot be assigned an arbitrary meaning (this point was first brought to my attention by Wilkins, 1977).</Paragraph>
    <Paragraph position="8"> Because of this requirement, portions of images of surfaces or objects (involving two or three dimensions) bear a one-to-one structural isomorphism to the corresponding portions of the referent. That is, portions of the representation correspond to portions of the referent, and the spatial relations between portions of the referent are preserved in the image. This property has been described by Shepard (1975) as In  this case, there is not a genuine first-order isomorphism, where a triangle is actually represented by something triangular in the brain, but there is a more abstract isomorphism where a triangle is represented by a set of representations corresponding to the vertices and sides standing in the proper relations. Thus, images depict, not describe '. while any symbol can be used to represent an object or part thereof in a description, the particular representation of such in an image is constrained by other representations--given that the interportion spatial relations must be retained in the image representation.</Paragraph>
    <Paragraph position="9"> The following three additional properties follow from our CRT metaphor: 3} Images occur in a spatial medium that is equivalent to a Euclidean coordinate space. This does not mean that there is literally a screen in the head. i Rather, locations are accessed such that the spatial properties of physical space are preserved. A perfect example of this is a simple two-dimensional array stored in a computer's memory: There is no physical matrix in the memory banks, but because of the way in which cells are retrieved, one can sensibly speak of the inter-cell relations in terms of adjacency, distance, and other geometric properties. 4) The same sorts of representations that underlie surface images also underlie the corresponding percepts.</Paragraph>
    <Paragraph position="10"> Hence, in addition to registering spatial properties like those of pictures, images depict surface properties of objects, llke texture and color. Thus, although the image itself is not mottled, or green, or large or small, it can represent such properties in the same way they are represented in our percepts.</Paragraph>
    <Paragraph position="11"> That is, the image representations must be able to attain states that produce the Qualia, the experience of seeing texture, color, size and so on.</Paragraph>
    <Paragraph position="12"> 5) Finally, by dint of the structural identity of image representations and those underlying the corresponding percept, images may be appropriately processed by mechanisms usually recruited only during like-modality perception, For example, one may evaluate an image in terms of its &amp;quot;size&amp;quot; (i.eo, being depicted--the representation itself is n~ither large nor small} in the same way one would evaluate the representation evoked while actually seeing the object, Images, then, share virtually all the properties of percepts, as opposed to properties of pictures or objects themselves. I refrain from making a i. Although there could be, if images occur as topographic projections on the surface of the cortexl this kind of space is a subset of the one I am defining here, however.</Paragraph>
    <Paragraph position="13"> complete identity because of a crucial difference: Perceptual representations are &amp;quot;driven&amp;quot; from the periphery, whereas images are somehow formed from memory.</Paragraph>
    <Paragraph position="14"> Hence, in both cases there may be particular kinds of &amp;quot;capacity limitations&amp;quot; that influence properties of the representation. For example (and this is an empirical question}, images may be coarser and less detailed than the corresponding percept because of memory capacity limits.</Paragraph>
    <Paragraph position="15"> These properties of images can be further understood in contrast to properties of &amp;quot;propositional&amp;quot; representations. Consider the two representations of a ball on a box illustrated in Figure I. A propositional representation must have: 1) a function or relation; 2} at least one argument; 3} rules of formation; and 4) a truth value.  relations only emerge from the conglomerate of the components being represented together. Thus, one needs two components before a relation like &amp;quot;on&amp;quot; can be represented.</Paragraph>
    <Paragraph position="16"> 2} Images do not contain arguments.</Paragraph>
    <Paragraph position="17"> The components of an image are not discrete entities that can be related together in precise ways. The box, for example, can be decomposed into faces, edges, and so on--and these are certainly not arguments in and of themselves.</Paragraph>
    <Paragraph position="18"> 3) Images do not seem to have a syntax (except perhaps in the roughest sense}. That is, a relation llke &amp;quot;on&amp;quot; requires two arguments in order to create a well-formed proposition; &amp;quot;on box&amp;quot; is an unacceptable fragment. In contrast, any syntax dictating &amp;quot;well-formedness&amp;quot; of pictures or images will probably depend on som e sort of interaction with a &amp;quot;semantic component,&amp;quot; will depend on what an image is supposed to be an image of.</Paragraph>
    <Paragraph position="19"> As we all know, &amp;quot;impossible pictures&amp;quot; are created regularly (e.g., by artists such as Escher), and rules that govern the nature of objects in the world may not  necessarily constrain the things that one can depict in a picture.</Paragraph>
    <Paragraph position="20"> 4) Fiy ally, unlike a proposition, an image does not have a truth value. In fact, as Wittgenstein (1953} pointed out, there is nothing intrinsic in a picture of a man walking up a hill that prevents one from interpreting it as a picture of a man sliding downhill backwards. The meaning of an image, and hence its truth value, are assigned by processes that work over the representation and are not inherent in the representation itself.</Paragraph>
    <Paragraph position="21"> 2.0 Five classes of empirical findings supporting the functional reality o~f visual mental image s</Paragraph>
  </Section>
class="xml-element"></Paper>