Audio tactile systems for designing and learning complex environments as a vision impaired person: Static and dynamic spatial information access

Nomad systems are designed to give blind people access to graphical and spatial information and have been developed in Australia. The age range of users to date is 5 years to 84 years. For the present users must be able to hear 'normally', if necessary using a hearing aid. A refreshable braille output capability is under development for totally deaf-blind people and should be available by the end of the year.

Four components of the system will be outlined.

1. The TouchBlaster Nomad (TM) CAD System
2. The TouchBlaster Nomad(TM) Kernel
3. The TouchBlaster Nomad (TM) Information Access System
4. The TouchBlaster Nomad(TM) Walkabout System

Each of these components has been designed for use by totally blind people but may be used effectively by sighted and partially sighted people.

It is said that, "A picture is worth a thousand words". If one is blind it should be helpful to be able to touch a picture or text and be told, if necessary in great detail and if possible in any language, what it is that is being touched. Such feedback will go a long way towards removing a serious disincentive to use of tactile graphics: the difficulty of interpreting what it is that is being touched, from moment to moment.

In a most general way the TouchBlaster Nomad systems involve interactive computer based procedures of a very straightforward type, that enable the making and use of tactile graphics to enhance understanding of the relative positions of entities, their scale and distribution, as a drawing, a three dimensional model or in the real space and real time, with movement.

"Perceptual information is stored in memory for future use. But information in memory is not a simple copy of perceptual experience, and memory is selective, suffers from retrieval problems and often distorts the original perceptual input" (Baddley, 1986). Much research has concentrated on spatial distortions in cognitive maps experienced by sighted people (Downs & Stea, 1973), and similarly there exists a considerable literature on the cognitive maps of blind people; studies by Drever (1955); Millar (1976); Dodds (1982) representing but a few examples.

In the case of the congenitally blind absence of previous visual experience appears to be a handicapping factor, although precisely what role previous visual experience plays is not universally agreed: some workers emphasising the role of visual imagery (Revesz, 1950; Juurmaa, 1973); others construing the problem as an absence of a visual frame of reference (Fisher, 1964; Warren, 1977), and yet others claiming that vision serves to integrate the other sensory modalities (Connolly and Jones, 1970).

A further possibility, not considered by others is that in the absence of vision the referent of the map itself (ie. the environment [usually a complex geometry involved]) may not be understood in a way compatible with comprehending sighted conventions. In other words, without the understanding of the "bird's eye view" convention, and the fact that a map uses space to represent space, the information carried by tactile maps may forever be incomplete or even mysterious to the congenitally blind individual. This, however, does not mean that useful spatial information cannot he obtained from tactile maps or that the congenitally blind should he denied the opportunity for using them (my emphasis)." (Dodds. 1988).

At the time Dodds was presenting these views the audio tactile approach to graphics development for the blind was still in prototype format (Parkes, 1988), but Dodds (1988) wrote, "Don Parkes' (1988) work on this topic is of considerable relevance." He was referring in particular to the information processing problems that are faced by blind people who are trying to process spatial (tactile graphic) and text (Braille) information, on the same medium. With the TouchBlaster Nomad audio tactile system no Braille is required, but can be used if desired. These considerations are significant to the second of the three components that we shall be considering.

i. The TouchBlaster Nomad CAD

TouchBlaster Nomad CAD [1] is a tool that enables totally blind and partially sighted people to create tactile and print drawings in a manner that is very similar to that which would be used by sighted persons when using a computer aided design (CAD) package to create a simple or a complex graphical image[2].

The Nomad system CAD (Can't Anyone Draw?) uses the Nomad touch pad and in built voice synthesis capabilities, linked to any IBM or compatible computer. The CAD software and a tactile icon template allow a blind person to select a range of regular Euclidean shapes, draw freehand and label the entities drawn so that when they are touched again the blind user is given a description. The user is able to draw lines of a declared length, and construct complex shapes, such as floor plans and route ways or personal images of a real world multi-function retail centre, using tactile icon commands to generate orthogonalised lines and diagonals of declared angle. A range of straightforward editing procedures are also available

The designs that are made can be embossed directly from the CAD system or saved as a file and input to another innovative Australian drawing package for the blind called Picture Braille(TM). This package allows detailed fat-bit editing of drawings by a sighted person and the addition of grade 1 or grade 2 Braille code to the drawing [3]. These editorial procedures do have to be undertaken by a sighted person however. But this is not a necessary extension of the TouchBlaster Nomad based system. It is however a most useful facility if a blind person is working together with a sighted designer because the sighted person's changes can be fed back to the blind designer by producing an edited embossed copy. This procedure operates as a positive feedback loop to each person involved in the design or behavioural environment.

How does such a design capability enhance learning of complex environments by blind persons? It is my conjecture, and that is all that it is at this time, that a blind person who is able to represent a personal image or "mental map" of an experienced complex environment, or behaviour setting, in tactile form will be able to seek a sighted person's guidance on the factually incorrect elements of the drawing. For instance a particular shop may have been represented as next to or opposite a feature which does not exist or which is in fact located in mirror location. The designs may also be drawn to scale in geometric or time units according to the wishes of the blind designer. The role of mental maps and tactile representations of space and the related behaviours of blind people has been most intensively studied in the work of Dodds (1977, 1980, 1988) and Dodds et al (1982).

Better understanding by sighted and blind people alike, of the manner in which blind people represent the three dimensional geometry of the world, through their CAD constructions will lead to improved architecture, for all. It has been my experience that no arrangements of the physical world that are an aid to the blind traveller become a handicapping factor to the sighted traveller.

The TouchBlaster Nomad CAD will have value at all levels of education and especially in the preparation of building plans and of schools, colleges and universities. In turn as user tactile graphicacy skills develop there will be a growing demand for tactile maps and plans of complex shopping centres, High street facilities and relate information to which it is the right of every blind person to have access. Spatial mobility leads to social and economic mobility and a human ecology that will rapidly converge to that of the sighted population. There is no longer any reason at all why every public body that can afford to print a map for a sighted person, cannot prepare and print a similar map or plan for use by people without sight: who almost certainly will put the same information to more intensive use!

ii. The TouchBlaster Nomad Kernel

In the kernel it is also possible to set the scale of maps and plans that are later used in components (iii) and (iv) below and to calculate straight line and irregular route distances, areas and the preparation and reading of line, bar and circle graphs. The kernel also allows the import of scanned text files which can be placed at any position on a map or plan and appropriately edited they may be "read" interchangeably with graphics, much as a sighted person moves from text to picture.

Other features in the kernel include a capability to paint with sound in nine different frequencies and also to draw and emboss a graphic, directly to an embosser or screen dump to a printer. If the printer has a carbon ribbon it is possible to print directly on to capsule paper and using a simple halogen lamp (150-300 watts is ideal) as used by video cameras, it is possible to 'heat' the image and get a tactile impression. An embosser such as Mountbatten or Versapoint is the quickest and easiest way to get a tactile output. The Mountbatten has the advantage of being able to use a wide range of papers, as it does not require a tractor feed.

Although there are about 75 commands to various functions in the kernel, including a simple notebook in to which text may be typed while in the TouchBlaster Nomad environment, these functions are not directly relevant to our objectives here.

iii. The TouchBlaster Nomad Information Access System

All sounds may be digitised into the map or plan and in this way all blind users, whether they have Braille skills or not, will be given information about the environment to which the map or plan relates, when they tactile surface is touched. A powerful feature of the system is the ability to use digitised sound. This means that maps and plans may be prepared in any language, with perfect speech output. It is also possible to record into the map or plan the actual in situ sounds that belong to a particular feature. In this way the dimensionality of the tactile graphic is increased to 4, at any point. Regular or periodic users of a centre can be given a disk copy of the graphic and a copy of the tactile map or plan and this can then be read at leisure at home or in a library or at any other place where there is access to the TouchBlaster Nomad system. In this way complex environments can be studied in preview or postview mode and the user is able to annotate the map with questions and aides memoires for personal guidance. This component relates closely to the last of the components that I wish to present in this short paper: TouchBlaster Nomad Walkabout.

iv. TouchBlaster Nomad Walkabout

The Walkabout component enables a user to place a tactile map or plan onto the Nomad pad and then trace a path along which a proposed trip is to be made. Such a trip may be no longer than from a bedroom to the toilets in a conference venue, or the trip from the residential accommodation to the lecture theatres and so forth.

'Walkabout' will take note of all points that lie alongside and adjacent to the route that is to be travelled. The adjacency criterion has to be set by the user, for instance wanting information on environmental features that lie within 50 metres of a point on the map or plan. The scale of the original map or plan is then used, together with a preferred walking speed (we disallow sprinting by blind people!) to produce a space-time audible dynamic facility, in which the user can listen to the planned journey in real time.

The information may be heard by playing it back on the computer that was originally used to create it or the journey may be saved to disk and taken to a another computer, with built in sound synthesis and accessible digitised sound and listened to at leisure. In this way the traveller is able to preview a journey, gain some idea of the audible environment through which the journey will pass, assuming that the original TouchBlaster Nomad map or plan had digitised sound integrated within its structure.

For a more dynamic use of the utility it is possible to carry a small palm top or lap top computer with appropriate sound output capabilities and be guided, turn by turn in real time. The same can be done, perhaps more conveniently by making a cassette recording of the 'Walkabout' journey and then taking this as a Walkman tape. While such use is certainly going to assist the learning of new and complex environments, the utility should obviously not be used as a guidance system in traffic zones or in any area where there may be danger.

It is likely to be particularly useful for students at all levels and it is my firmly held view that this is a critical application of the utility because it extends the reach of the blind student, gives him or confidence to talk about the school, college or university: enables the development of 'mental maps' of the educational environment. How these mental maps, generated by the audio tactile plan or map are actually used is much less important than that they now exist and that means there is a potential for use.

There is a sense in which this last component might be described as a virtual travel system. without too much stretch of the imagination!

Summary

Acknowledgments

To Richard Dear, my sincere thanks as always for his friendship and programming skills. During the past seven years during which these and related components of the concept of audio tactile graphics have been under development he has given many hours, days and weeks of his spare time to programming the components, often under the most difficult circumstances as I said, "What if the system....?" Should you ever meet Richard, please be careful not to say, "What if....?"!

Footnotes

1. For information on prices and the availability of TouchBlaster Nomad and related systems, please contact:

   Australia, New Zealand, Asia and Africa
   Quantum Technology Pty Ltd
   5 South Street
   Rydalmere 2116
   AUSTRALIA
   Fax + 61 (0)49 684 4717 Ph. + 62 (0)49 684 2077

   Canada, USA, Central and South America
   American Printing House for the Blind
   1839 Frankfort Ave
   PO Box 6085
   Louisville KY 40206-0085
   USA
   Fax + 1 502 895 1509 Toll Free (USA) 800-223-1839

   Europe
   Queen Alexandra College for the Blind
   Court Oak Road
   Harborne
   Birmingham B17 9TG
   ENGLAND
   Fax + 44 (0)21 428 2282 Ph. + 44 (0)21 428 5050

   AVH
   Assoc. Valentin Hauy pour le Bien des Aveugles
   5 Rue Duroc
   75343 Paris Cedex 07
   FRANCE
   Fax + 16 (1) 4567 8978 Ph. + 16 (1) 4567 4945

2. Golledge (1991) has written that, "Until the development of a technical device called NOMAD, blind or visually impaired people could not use most maps and graphics." Golledge is Professor of Behavioural Geography at the University of California, Santa Barbara. Before the onset of his blindness in 1985 his research on way finding strategies and environmental perception was well established. He has continued to pursue the same field of research at UCSB but now with particular emphasis on way finding in complex environments, by blind people. He is working with Professor Jack Loomis of the Department of Psychology to develop and satellite based guidance system that uses a virtual acoustic device, developed at UCSB. Delegates with an interest in spatial information access and mobility for blind travellers will benefit greatly by contacting Professor Golledge at UCSB.

3. Picture Braille is produced by OPTEK Systems Pty. Ltd. of Dundas, Sydney, Australia. In the USA it is distributed by American Printing House tor the Blind.

References

Connolly, K. & Jones, B. (1970). A developmental study of the afferent-reafferent integration. British Journal of Psychology, 61, 259-266.

Dodds, A. G. (1978). Hemispheric differences in tactuao-spatial processing. Neuropsychologia, 16, 247-250.

Dodds, A. G. (1980). Spatial Representation and Blindness. Unpublished PhD Thesis, University of Nottingham, Nottingham, England.

Dodds, A. G. (1988). Tactile maps and the blind user: Perceptual, cognitive and behavioural factors. Proceedings of the Second International Symposium on Maps and Graphics for Visually Handicapped People. Dodds, A. G. & Tatham, A. F. (Eds.) King's College, University of London, 30-35. For copies please contact Dr. A. F. Tatham, Royal Geographical Society, 1 Kensington Gore, London, England.

Dodds, A. G., Howarth, C. I. & Carter, D. D. C. (1982). The mental maps of the blind: the role of previous visual experience. Journal of Visual Impairment and Blindness, 76(1), 5-12.

Downs R. M. & Stea, D. (1973). Image and Environment. Edward Arnold, London.

Drever, J. (1955). Early learning and the perception of space. American Journal of Psychology, 68, 605-614.

Golledge, R. G. (1991) Tactual strip maps as navigational aids. Journal of Visual Impairment and Blindness, 85(7), 296-301.

Fisher, G. H. (1964). Spatial localisation of the blind. American Journal of Psychology, 77, 2-14.

Jansson, G. (1983). Tactile maps as a challenge for perception research. Proceedings of the First International Symposium on Maps and Graphics for the Visually Handicapped, Wiedel, J. (Ed.), Assoc. Amer. Geogs. Washington.

Millar, S. (1976). Spatial representation by blind and sighted children. Journal of Experimental Child Psychology, 21, 460-479.

Parkes, D. N. (1988). An audio-tactile device for the acquisition, use and management of spatially distributed information by visually impaired people. Proceedings of the Second Symposium on Maps and Graphics for Visually Handicapped People. Dodds, A. G. & Tatham, A. F. (Eds.) King's College, University of London, 30-35.

Parkes, D. N. (1990). NOMAD graphics for blind children. Papers and Proceedings of the Australia and New Zealand Association for the Education of the Visually Handicapped, Homai College, Auckland, 6-12.

Parkes, D. N. & Dear, R. J. (1991). Making and using high resolution audio-tactile orientation and mobility plans and maps with the NOMAD system. Papers and Proceedings of the 6th International Mobility Conference. Madrid: The Spanish National Organisation of the Blind, 1, 329-336.

Parkes, D. N. (1991). Nomad: Enabling access to graphics and text based information for blind, visually impaired and other disability groups. Conference Proceedings, World Congress on Technology. Arlington, Virginia, 5, 688-714.

Revesz, G. (1950). The Psychology and Art of the Blind. Longmans Green & Co. Ltd., London.

Warren, D. H., Anooshian, L. J. & Bollinger, J. G. (1973). Early vs. late blindness: The role of early vision in spatial behaviour. Research Bulletin, American Foundation for the Blind, 26, 151-170.

Author: Don Parkes, Director, Ecology of Blindness and Audio Tactile Graphics Research Unit, Behavioural Sciences - Medicine, University of Newcastle NSW. Please cite as: Parkes, D. (1994). Audio tactile systems for designing and learning complex environments as a vision impaired person: Static and dynamic spatial information access. In J. Steele and J. G. Hedberg (eds), Learning Environment Technology: Selected papers from LETA 94, 219-223. Canberra: AJET Publications. http://www.aset.org.au/confs/edtech94/mp/parkes.html