Visualization by People without Vision 
Vladimir BULATOV 
Dept. of Physics and Science Access Project 
Oregon State University 
Corvallis, OR 97331-6507 USA 
bulatov@dots.physics.orst.edu 
John GARDNER 
Dept. of Physics and Science Access Project 
Oregon State University 
Corvallis, OR 97331-6507 USA 
gardner@physics.orst.edu 
103 
Abstract 
Prototypes of two direct audio/tactile access 
methods are demonstrated that allow people 
with print disabilities to "see" diagrams, maps, 
charts, and other graphically-displayed 
information. All methods require that labels or 
other identifying information be available for 
each important object in a figure. In web 
applications, this additional information is 
accessed by exercising a link associated with an 
object. People with severe visual disabilities 
may need a tactile image mounted on a touch 
screen to access object-related information. It is 
expected that recently-introduced force 
feedback "haptic mice" may soon provide easy 
haptic access to the objects as well. The critical 
factor that ensures accessibility is availability of 
information about all important objects, since 
there is currently no way to transform any but 
the simplest purely visual graphic information 
into audio or tactile/haptic information that is 
easily understandable. These requirements, 
which also ensure good searchability, should 
influence development of future data structures 
and visualization displays. 
Introduction 
The mission of the Science Access Project 
(SAP) is to develop technologies that permit 
non-visual access to more complex electronic 
information. Until recently this research has 
concentrated largely on access to tables, math 
equations and other character-based 
information. Access is possible if this 
information is presented in some good mark-up 
language such as SGML and difficult to 
impossible when presented as bit-mapped 
images or visually-formatted text. 
More recently the SAP has begun exploring 
possible methods that permit good non-visual 
access to information conventionally presented 
as graphics. 
1 Methods for Non-Visual Access to 
Graphical Information 
There are a number of audio technologies that 
are useful as substitutes or enhancements for 
visual presentation of graphical information. 
Many of these technologies rely on feedback 
from the computer to identify and display 
information about the important objects in the 
figure. 
Blind people have good access to many 
computer applications through use of screen 
reader software and either a speech synthesizer 
or a braille display. Speech technologies have 
proven to be useful to many people who are not 
blind. Many people suffer from visual dyslexia 
sufficiently that they can understand virtually 
nothing in prrint even though they may have 
perfect vision. It is also well understood that a 
substantial fraction of people are "audio 
learners" who understand what they hear better 
than what they see. All these people benefit 
from availability of speech and other audio 
output instead of or in addition to the normal 
visual display. 
However speech or other alternate access is 
presently limited largely to unformatted text. 
It is found, for example, that blind people can 
"read" maps rather well by using a touch screen 
and running their fingers along a road or 
railroad track, interrogating cross streets as they 
are encountered, etc. (Jacobson, in press). This 
"map-reading" method requires two things: 
1. The computer must have some means of 
disclosing to the viewer the objects at any given 
location, and 
2. the blind person must be capable of 
assimilating a spatial image of where various 
objects lie on this map. 
A common technique used to make assimilation 
of the figure easier is through use of a tactile 
image. Unfortunately there are no technologies 
for displaying refreshable tactile images on-line, 
104 
but it has recently become possible to print a 
tactile figure, place this figure on a touch screen 
or other digitizing tablet, feel the tactile images, 
and receive audio feedback from the computer 
about those images. 
The most convenient tactile printing technology 
is the TIGER Tactile Graphics EmbosseR 
(TIGER) developed in the SAP group. One can 
also print or copy a black and white image on 
"swell paper" (Swell), heat this special paper in 
a radiant heater, and obtain a tactile copy when 
the black areas swell. 
Off-line printing is time-consuming and makes 
it very difficult for a blind person to take 
advantage of features like zoom views. A 
number of on-line haptic technologies are being 
explored motivated by various virtual reality 
uses. One such technology, the haptic mouse, 
has already been introduced or announced for 
imminent release by several companies 
(Immersion, Canada 1, Canada2). These devices 
should soon permit blind people to explore 
graphics on-line. 
Another audio technology is use of non-speech 
audio to display visual images and other data 
(ICAD). The TRIANGLE application 
(TRIANGLE) and a new Windows 95 
Accessible Graphing Calculator (ACG) both 
developed by the SAP uses tone plots to 
represent x-y graphs. The x axis is mapped to 
time and the y axis to a tone that is high for 
large y and low for small y. This provides a 
good qualitative overview of a graph. The SAP 
is currently investigating audio enhancements to 
provide more quantitative non-speech audio 
information such as beats with frequency 
proportional to the slope or second derivative of 
y with respect to x (Audio). 
People with only moderate visual disabilities or 
people with good vision but various visual 
processing disorders (e.g. dyslexia) can achieve 
very good access by the combination of visual 
images reinforced by audio, and many are 
expected to find the haptic mouse useful as well. 
The critical necessity for any of these methods 
to work is a well-structured data format that 
includes labels and permits other information 
about objects and the location of these objects in 
the figure. 
2 Accessibility of bit mapped graphics 
on the WWW 
A bit map image is essentially inaccessible, 
because the visual appearance can seldom be 
translated into an audio or a tactile image that is 
easily recognized by a person with a severe print 
disability. 
The SAP has recently developed a relatively 
simple method that permits authors or editors to 
"make bit maps accessible" by adding 
identifying information related to objects in the 
figure. This procedure requires a current 
generation 4 Netscape or Internet Explorer 
browser and a screen reader that works with 
those browsers. An example (Example) is a 
campus map of Oregon State University. If the 
user clicks on the "accessibility link" adjacent to 
the bit map picture, the image is moved to the 
top frame of a double frame window. As the 
mouse pointer is moved from object to object, 
the label is displayed in the bottom frame. The 
screen reader automatically triggers on any 
change and reads the labels as they appear. The 
user may freeze the label by clicking on the 
object. This permits long or confusing labels to 
be browsed by the screen reader. Additional 
information may be linked from this window of 
course. 
A blind user can explore the image with a touch 
screen instead of a mouse. Ideally she/he will 
have a tactile image on a digitizing pad. In the 
future it should be possible for a blind user to 
browse the picture on line with a haptic mouse. 
This access method works for a blind user even 
if the internet browser is set not to show 
pictures. With "no picture" enabled, the image 
will not appear in the top frame visually, but the 
105 
labels still appear when the mouse pointer is 
placed where a particular object would be if it 
were visible. 
Preparation of the information necessary to 
work in the accessibility link is relatively 
straightforward. The author or an editor must 
outline each object and write its label 
information. The only caveat is that authors 
must use the correct order for listing objects that 
overlap to be sure that the right one is on top. 
The author/editor tools will be tested by Oregon 
State University faculty and staff and will be 
used to make a number of local and distance 
education courses more accessible beginning in 
Fall 1998. If these do prove very useful we will 
encourage their use in education more broadly. 
3 VRML as a Graphics Language 
The requirements that make information directly 
accessible in principle by blind people are 
basically the same as those that make 
information most usable by search engines, 
visualization routines, etc. 
Virtual Reality Modeling Language (VRML) is 
a high-level well-structured language permitting 
very flexible modeling of three dimensional 
time-dependent interactive graphics. There are 
several research efforts aimed at making VRML 
more accessible by non-visualmeans (Toronto, 
NIST). 
VRML may not be intrinsically fully accessible 
but it is a language capable of representing 
graphical information in a fully accessible form. 
To our knowledge there is no other common 
graphical language with this capability available 
at this time. 
We have constructed several VRML examples 
(VRML) to illustrate graphical information 
common in physics education. We discuss the 
accessibility of each and how future 
developments could increase accessibility. 
One example is an interactive display of electric 
potential along a line containing electric 
charges. The sign, magnitude, and position of 
the charges may be changed by the user. 
Another example is a familiar picture from an 
elementary physics textbook illustrating friction 
on a block sliding down a tilted plane. This 
example is highly interactive, allowing the user 
to change the coefficients of friction, control the 
angle of tilt, etc. 
The initial static pictures are quite accessible 
using technologies that are available today. 
Sighted viewers who want to see the labels can 
click on any object, and a dialogue box pops up 
and displays any information that the author has 
included about that object. The labels are 
included in nodes using a procedure similar to 
that used by Ressler et al. (NIST). If the user is 
also running a speech screen reader, it will 
normally speak the contents of this dialogue 
window, so the label will be both seen and 
heard. 
A totally blind reader can print a tactile copy of 
the figure, place it on a digitizing pad, and then 
use her/his finger as the mouse to call up and 
hear the dialogue box provide information about 
objects. A deaf blind user can access the 
dialogue box with an on-line braille display. 
We anticipate that haptic mouse developments 
are likely to make possible direct haptic access 
to these figures within a year or two. A 
combination of a static tactile picture and a 
haptic mouse could provide really excellent 
access to these examples. Feeling out a picture 
with a haptic mouse is tedious, so the tactile 
figure is good for a general overview. Then the 
mouse can be used to examine changes in the 
potential for example, or to detect the moment 
the block begins to slide as one changes some 
parameter. 
The important information in the electric 
potential example could also be displayed in 
audio by the same technique used in the SAP 
Accessible Graphing Calculator (AGC). The 
106 
advantage of an audio display is that the user 
can easily change something and then listen 
again. It would be tedious at best to read 
different figures by making tactile copies Of 
each. This type of simple tone plot can also be 
displayed as a moving icon on a braille display 
for a deaf blind user. This technique is used in 
parallel with the tone plot in the TRIANGLE 
program. 
The SAP is making several tools that permit 
audio display to be used conveniently with 
VRML figures. These tools include a user- 
controllable method for choosing objects to be 
displayed in a tone plot. The SAP is also 
making a simple authoring tool for x-y graphs in 
VRML. These data could be displayed in audio 
and examined quantitatively in the same display 
routine developed for the SAP Accessible 
Graphing Calculator. This plot browser 
provides excellent on-line audio or braille/haptic 
access to graphs. It can search for maxima, 
minima, inflection points, etc., and can speak 
the data values at any point. 
It would not be difficult to extend this simple 
graphing tool to an almost arbitrarily rich and 
complex graph authoring environment with 
data, functional forms, and other types of 
information available to the viewer in both 
visual and alternate form. 
VRML is a three dimensional modeling 
language, but at present there is no fully 
satisfactory way that a blind person can view 3D 
images. Commercial interests are likely to drive 
development of technologies for feeling virtual 
objects however. These could bring excellent 
access by people who are blind or dyslexic to 
properly prepared VRML graphics. We suspect 
that clever audio methods are possible that 
provide some degree of access even in absence 
of tactile hardware. There is much interest in 
spatially localized sound, and audio access to 
three dimensional displays by blind people 
could also become possible as a result. 
However all these access methods require that 
important objects be labeled. 
Conclusion 
We have demonstrated a "band-aid" approach to 
making current bit-mapped graphics files 
accessible. This method requires adding an 
object list to the original bit mapped image that 
relates the position of objects to their labels. 
These labels may be accessed in standard web 
browsers and spoken or displayed in braille by a 
standard screen reader. 
We have also demonstrated several examples of 
accessible VRML graphics. 
We intend to make authoring and editing tools 
available that allow others to produce label 
nodes for VRML objects and to create certain 
kinds of graphics (e.g. x-y VRML graphical 
data displays) that will automatically include all 
necessary information. 
Most general VRML images can be made 
accessible in principle if the author takes the 
time to label objects so that they are accessible 
to the audio browser. 
Full access to simple graphics (i.e. two 
dimensional time-independent displays) is 
possible now. Good access to more complex 
displays is possible in some cases with difficulty 
but requires development of better tactile or 
haptic hardware to permit really good access. 
Even in absence of such hardware however, we 
have demonstrated that clever users can achieve 
a degree of access that is almost beyond the 
imagination of most people today. 
The major lesson of this research for computer 
scientists is that data visualization by people 
without vision is possible only if the data 
structures are high level and include adequate 
information. Visual appearance is generally 
inadequate to permit full identification of an 
object by people with visual disabilities, so 
explanatory labels are required. In addition the 
data structure and data display routines must 
107 
relate the object to its position and vice versa. 
These properties are desirable in any case, 
because they greatly facilitate classification and 
search methods both present and future. 
Acknowledgements 
This research was supported in part by the 
National Science Foundation under grant 
HRD9452881. 

References 
SAP, The Oregon State University Science Access 
Project, http://dots.physics.orst.edu 
Jacobson, R.D. and Kitchin, R.M (in press, due 1998) 
Geographical information systems and people with 
visual impairments or blindness: Exploring the 
potential for education, orientation and navigation. 
Transactions in Geographical Information Systems. 
TIGER., The TIGER Tactile Graphics EmbosseR:, 
http://dots.physics.orst.edu/tiger._.proj ect.htrnl 
Swell, Major manufacturer of swell paper in the US 
is Repro-Tronics, Inc., 75 Carter Ave., Westwood, 
NJ 07675 http://www.repro-tronics.com 
Immersion, Immersion Corporation, San Jose, CA, 
http://www, force-feedback.com. 
Canada I, Control Advancements, Kitchener, Ontario, 
CANADA, http://www.controladv.com 
Canada:, VisuAide, Longuauil, Quebec, CANADA 
ICAD, International Conferences on Audio Display, 
http://www.santafe.edu/-kramer/icad/ 
TRIANGLE, TRIANGLE, a computer application 
enabling blind people to do math and science, 
http://dots.physics.orst.edu/triangle.html 
AGC, Windows 95 Accessible Graphing Calculator 
http://dots.physies.orst.edu/age.html 
Audio, Audio graph research in the Science Access 
Project 
http://dots.physics.orst.edu/audiodisplay.htrnl 
Example, Making WWW bit map graphics accessible 
http://dots.physics.orst.edu/graphics bitmap.html 
Toronto, Christopher Serflek, Jutta Treviranus, - 
VRML: Shouldn't Virtual Ramps be Easier to Build 
http://www.utoronto.ca/atrc/rd/library/papers/vrml. 
html 
NIST, Sandy Ressler, Qirning Wang, Making 
VRML Accessible for People with Disabilities 
http://ovrt.nist.gov/projects/VRMLaccess/ 
VRML, Accessible Graphics with VRML 
http://dots.physics.orst.edu/graphics_vrml.htrnl 
