|
The paper will describe a number of public information and educational multimedia projects undertaken by the author over the last two years from the perspective of a visual communication designer. Attention will be focused on the problem of creating graphics for multimedia applications, and the application of various software tools and techniques for this purpose. Work discussed will include projects for the 'Dinosaurs Alive' exhibition held at the Scitech Discovery Centre in Perth, Western Australia during 1992-93, projects related to North West Shelf Gas for both Woodside and the State Energy Commission of Western Australia, ALEC the Aboriginal Legislation Education Console for the Francis Burt Law Education Centre at Fremantle in Western Australia, and the Perth Mint Information Base, a multimedia based visitor information system.
Multimedia delivery technology is currently at a point where many of those previous constraints both technological and economical to its wider application will no longer be an issue. The introduction of consumer priced 'TV top' CD/MPEG based interactive replay systems capable of full colour, full motion interactive audio visual replay such as promised by Commodore's CD32 and the 3DO based systems means that poor production values will be seen as precisely those, rather than the artefacts of technological limitation as has often and inevitably been the case to date.
It is therefore of some concern that almost all of the discourse and commentary within the author's experience to this point, dealing with multimedia as an educational, marketing or information tool, and the research endeavour seeking to enlighten issues of knowledge organisation and navigation, the relative efficacy of constructivist versus didactic approaches and the metaphorical representation of human activity for the purpose of screen based knowledge interaction, etc., treats the essential point of contact between the end user and the delivered content, the user interface, as merely the functional application of scientifically definable ergonomic considerations. The failure to account for, acknowledge or it seems often even recognise the significance of emotive influence on communication efficacy and quality of experience for the end user resulting from decisions or their absence regarding detailed attention to subtleties of the audio and visual environment presented within a multimedia experience, poses a question rarely addressed as a significant variable by researchers in this field. Just as in human interaction it is nuance in gesture and demeanour or subtle inflection and intonation which can dramatically alter and determine the interpretation of meaning in communication; so in audio visual media, the quality of the soundscape, variance in shape, texture, colour, arrangement, abstraction, representation, typography, kinetics and timing can and anecdotal evidence certainly indicates do, predispose user attitude toward content, and therefore, the author asserts, dramatically impact and vary the quality and quantity of communication achieved for different demographically and psychographically defined users.
The enthusiastic satisfaction which currently we as multimedia producers gain from our ability to manipulate the content within the narrow constraints imposed by software and hardware parameters and not least our creative abilities, is we tend to forget, irrelevant to the visually sophisticated interpretation of the average lay end users of our products, who, rightly oblivious to the leading edge nature of our technology mediated achievement, translate their multimedia experience merely into a perception of poor quality television compared to the 'real thing' even if it is interactive, or worse, relative to the engagement and excitement available from commercially produced entertainment software employing the same delivery hardware. It is these issues of qualitative media sophistication which need to characterise future debates on multimedia rather than technological tricks, specifications and 'future' capability potentials which generally dominate multimedia dialogue today .
It is probable that industry stratification will occur, in much the same way as it currently exists in the world of television, video and film production, ranging from amateur through art, educational, industrial to full broadcast, as multimedia producers seek to align themselves to particular levels of genre, technical, or aesthetic value category, similar to those evident among their mainstream media counterparts. The amateur enthusiast, early adopter culture, a normal and necessary phenomenon of technological innovation will give way at the upper level to a multimedia publishing industry aware that the combined creative and production skills it relies upon, go well beyond those of any of its constituent media types alone.
The work to be discussed spans the preceding two years from a point where for the author at least, technological capability was the major problem area, to the present where production logistics and creativity issues dominate. Being undertaken as professional practice in support of teaching and multimedia design course development in the School of Design at Curtin University, the author's function within the multimedia creation process has also seen a shift of roles from that of graphic designer to that of producer, director, programmer and graphic designer. Apart from the media design base already mentioned, prior to this period, the author had benefited from working on three relatively major multimedia projects between 1987 and 1992, with Anthony Temple of Temple Interactive Media; The Ningaloo Videodisc Project, The Drilling Game, and the Videodisc based Language Partner Japanese 1. (Winfield & Temple, 1989), (Temple, 1990), (Temple, Pinfold, Latchem & Fox, 1991), (Temple & Borzyskowski, 1991).
The discussion also addresses the recent evolution of the Commodore Amiga computer which has been the authors preferred multimedia production and delivery platform to date. Indeed, on a cost performance basis, where direct PAL video compatibility or direct video and audio integration capability is concerned, there has until recently been very little if nothing else to choose from. Good quality scan rate converters for MAC or PC have not in the past been renowned for their low cost. Full screen video animation with built in 4 channel 8 bit stereo audio integration has also been until recently a feature unique to this platform. This has been available since the machine's launch in 1985 due to the included audio and graphics co-processor hardware, as has the true multi-tasking operating system, Amiga DOS, originally written by British, Bath University computer scientist, Dr Tim King, providing both a point and click graphical user interface an a DOS like command line shell. The multi-tasking environment allows for full inter-process communication by means of the AREXX interpreter which can co-exist with multiple simultaneously running unrelated software applications.
While in 1986 it was the unique PAL video compatible audio visual and animation capabilities which attracted the author away from the then audio visually sterile PC AT and Macintosh environments of the day, it is the multi-tasking capability which affords substantial productivity gains as well as enhanced multimedia delivery options today. For a graphic designer/multimedia author there are substantial advantages in being able to run the equivalents of Photoshop, Freehand and Authorware simultaneously and to be able to switch between them instantly to scan and process an image for example, or to adjust a structured drawing and then instantly switch back to check the results. Similarly, in multimedia delivery the ability to transparently communicate with or control external devices during an interactive audio visual presentation opens up a wide range of possibilities and has allowed the solution of sometimes tricky problems as will be described.
The projects which are the subject of this paper are in fact single delivery point 'kiosk' type applications so really the genealogy of the delivery platform is irrelevant except that it provide the most effective cost/performance solution to the communication task. Computer user brand loyalty frequently clouds this issue in the multimedia world which in itself may well become academic in the foreseeable future as one or other of the announced CD video player technologies comes to dominate the consumer market finally making available high quality multimedia delivery without the paraphernalia or cost of workstations.
The author's brief was to produce the required animation for a lip synchronised 3 minute monologue with a budget more appropriate to around 15 seconds of conventional cel and film based animation. Similarly single frame video recording of the approximately 2,500 required animated images was impractical on the budget available. The remaining solution was to employ multimedia technology for real time playback of the complete sequence directly from computer to video tape. Production of the animated artwork was equally impractical employing conventional processes.
In resolving these difficulties the authors approach was to identify software tools which were most closely able to emulate the various stages of production employed in traditional animation. These stages are soundtrack analysis and annotation according to phonemes against a predetermined constant playback frame rate, the application of this information in the production of artwork, in this case of mouth movements and (limited) facial expressions, and finally the marrying of the soundtrack with the completed animation. While there have of course been demonstrated several experimental systems which provide automated graphic linking and speech generation from text files to computer generated facial mannequins, the provided soundtrack, visual specification, production schedule and the afore mentioned budget constraints precluded the exploration of these, for the time being, somewhat esoteric possibilities.
A crucial factor to the success of the chosen approach was the ability of the production and playback computer to achieve and more importantly maintain a constant full screen audio and video playback rate of 12 images per second as is commonly employed in conventional cartoon animation. A standard 6820 processor equipped Amiga 2000 was capable of this with a choice of either SCALA or Showmaker software capable of preloading following segments as playback occurred to achieve a seamless full screen presentation to video.
The procedure followed was first the digitisation of the supplied narration audio track at around 15 kHz sample rate; then, employing audio software which represents the audio oscillograph fashion against a timeline on screen, noting the precise time, to 1/100 of a second, of each significant phoneme, consonant and vowel which determines mouth shape. This was done for the whole narration, the multiplication of the noted time by 12 providing the exact frame number of the animation on which any particular phonetic event occurred. This is equivalent to producing 'bar sheets' in film animation which the animator uses to create animation which matches the soundtrack.
Because the triceratops image was to be projected onto a fixed 3D model in the exhibit, artwork production for the speaking face needed only to be limited to the mouth and eyes within an otherwise static visage. This factor made it possible to accomplish the substantial duration of the animation within the limited time and budget available. The 'DeLuxe Paint' bitmap drawing program for the Amiga permits the setting up of hundreds of full screen animation frames which can be stepped through individually, painted and then played to check results. A page was created containing all the required mouth position drawings with transitional positions for all the phonemes as well as the dinosaur's eyes. This was kept on the program's spare page buffer providing instant toggled access irrespective of the animation frame number being worked on. Working with the frame number annotated script the animation was completed in batches of two hundred or so frames at a time corresponding to the audio track sentences, simply by selecting mouth parts from the spare page and clicking them onto the face which had already been drawn and copied to all the frames. Eyes with movement were similarly stencilled into position.
All that remained was to cue the completed segments aligned with their respective audio segments within one of the presentation programs mentioned and play the result at 12 frames per second to the video tape which was required for projection onto the 3D model. Multimedia techniques thus allowed the completion of the project in around 1/5 of the time normally required with zero expense on artwork materials, camera hire or film costs, etc.
The other two projects for which the author was commissioned to design and produce the graphic user interface content for this exhibition were 'Dinoquiz' and 'Designosaur'. These were two interactive computer based activities, the former of which allowed exhibition visitors to test the knowledge they had gained from the 'Dinosaurs Alive Exhibition' and the latter 'Designosaur', permitted visitors to specify by means of track ball, when their dinosaur lived, what it ate, and how large it was; at which point the screen displayed the resulting animal and provided the opportunity for users to enter their name, and watch as their personalised dinosaur image was plotted for them to keep. The interactive exhibits here were based on a PC delivery platform and employed custom coded software written by staff of the Scitech Discovery Centre, however. being exhibits aimed at children, the interaction response time needed to be rapid and therefore required all screen graphics to be preloaded into 16 MB of RAM when the machines were switched on. In order to achieve this, the software author specified PCX screen graphics limited to a fixed 16 out of 256 colour VGA palette, while the content authors specified the need for bright cheerful colourful images.
The screens incorporating both text and graphics were developed and produced using 'DeLuxe Paint IV' on an Amiga computer. The choice of colours for the fixed 16 colour palette were at the discretion of the author which provided the opportunity for the creation of colourful richly textured effects by means of pixel dithering. On completion the images were converted to the required VGA PCX format using 'Art Department Professional' image processing software and saved on PC formatted disks. The exhibits have enjoyed considerable success, 'Designosaur' having generated in excess of 40,000 output plots for users and at the time of writing being in use both at Perth's Scitech and in Hong Kong. A point worth noting when producing screen graphics for potential international distribution is to produce screens without text separately from the text which can subsequently be overlaid.
Scitech Resources project Manager, Denharn Dunstal had, in consultation with the client, determined the physical layout of the exhibit to incorporate a waist high touch screen console/kiosk, a large rear projection video display to echo the images appearing on the console monitor, and a requirement for the multimedia software to operate a series of QI spotlights to illuminate relevant parts of the engineering model at specific times during the multimedia presentation. A low cost, 68020 processor equipped Amiga 2000 with 4 MB RAM, coupled with a touch screen laminated Sony 21 inch PVM 2130QM PAL RGB monitor was chosen as the delivery platform for the project. The Amiga 2030 internal genlock provided the secondary composite signal for video projection, and additional electronic relay hardware was designed and built by Denham Dunstal to operate the spotlights.
It was the multi-tasking operating system of the Amiga line of computers which afforded a relatively straight forward solution to the problem of individually controlling the eight external spotlights while simultaneously running the audio and visual multimedia presentation. Both the final software implementation written by the author using the Director 2 scripting language and an earlier implementation prepared by Anthony Temple of Temple Interactive Media using CTS Designer, permitted the launching of external tasks from within the main program utilising the AREXX inter-process control language made resident during machine startup. Consequently, a series of short AREXX scripts coded by Nigel Headlam of Headlam Computers, which addressed the computers parallel port to which the lighting interface was connected were able to be launched whenever required during the multimedia presentation.
The final presentation relies upon a digitised recorded narration soundtrack as opposed to screw text for content elaboration. The author feels that this is a far more elegant exploitation of multimedia capability than the frequently seen screen text with sound effects approach or limitation to concurrent audio only when Quicktime is running. Furthermore the use of a continuous soundtrack frees the screen of text clutter, allowing grater scope for imagery, diagrams and animation. The author also found that linking the tasks of graphic design and software authoring also provided distinct advantages when fine tuning the final version of the program. In this instance however, where there was less than 2 weeks available for production between the time when the final script and narration soundtrack were made available and the exhibition opening, the author's prior team working relationship with Anthony Temple who at short notice produced the CTS Designer based software also had its advantages.
A final point worth noting with regard to this project was the adoption of the technique of preloading graphic screens which appear immediately after user interaction into RAM prior to requirement, during playback of a preceding sequence, in order to dramatically speed up interactivity; also the use of audio spooling from hard disk during replay, in this case obviating the need to load entire samples often in access of 1 MB into RAM, again dramatically improving responsiveness for the end user.
The State Energy Commission exhibit currently located at its intended site, a SECWA Visitor Centre at Karratha in WA, fulfils the brief of providing information concerning the pipeline delivery of gas from the processing plant at the Burrup Peninsula in the north west of Western Australia to industrial and domestic users in the south of the state as well as Carnarvon and Geraldton along the way. The exhibit, designed by Denham Dunstal of Scitech Resources to the production specification of Derek Longhurst of Channel 5, incorporates in addition to the multimedia component a number of large back lit photographic transparencies which are individually illuminated at strategic times during the presentation. The external control and software techniques adopted for the previously described project proved equally effective in this instance. Graphic design, production and software were the responsibility of the author, the major technical difference being the use of buttons below the screen for user interaction rather than touch screen.
An intercultural project commenced during the Year of Indigenous Peoples, ALEC provides a valuable addition to the educational experiences gained by parties the of high school students who are the main users of the centre, since in addition to educational activities including participation in mock trials and other tutorials which the centre provides, the multimedia computer is able to present and address in an easily accessible, audio visual form, information about legislation which has been enacted since European settlement of Western Australia with emphasis on how it has affected our indigenous Aboriginal population.
The software structure of ALEC has been developed in modular form so that while there are three study topics available on the system at the time of writing, "Heritage & Culture", "Rottnest Island as a Penal Settlement" and "Royal Commissions" , additional units such as "Pearling Laws" currently under development, can be very easily added with only minor operating script and main menu modifications.
A number of performance attribute specifications were identified at the start of the project which included turnkey operation, ie. switch on at wall socket in the morning and switch off at night, ease of use ie. no learning curve, visually engaging and responsive 50 that young users do not get bored, physically self contained and "bullet proof" to withstand continuous use by numerous groups of high school students, able to be used by up to two seated users with provision for a class group to be able to see, and software and hardware robustness. These required attributes wore accommodated by the specification of a delivery system consisting of an Amiga 4000 (68040 25 MHz) computer equipped with 6 MB RAM and a 340 MB hard disk providing both the required data storage capacity and transfer speed, driving by means of Fordray PAL encoder a Sony PVM 2130QM 21 inch PAL RGB stereo primary monitor and a Mitsubishi CT-2862AS S-Video 26 inch colour TV as the secondary audience monitor, all mounted within a steel framed custom built cabinet incorporating the two monitors, space for student worksheets and a seat. User interaction is facilitated by means of 5 game style buttons mounted beneath the primary monitor. The multimedia authoring system selected in this instance was SCALA, the sister program and functionally similar to the proven robust INFOCHANNEL, a public information software system used widely in the international arena by major hotel chains, cable television networks and major corporate users including Philips, the Swedish Parliament, Ericsson Telecom, SAS, Ford Motor Company, Renault, Esso and others. At the time of writing the installed multimedia hardware/software system has completed 8 months of daily use without a single failure.
The content presentation is again characterised by the use of voice over synchronised to and in conjunction with still and animated images although with the advent of the Amiga Advanced Graphic Architecture, up to 256,000 out of 16 million colour palette are now available in any resolution on screen without additional hardware. This factor has permitted the author to utilise 24 bit paint and image processing software to develop screen graphics. Access to the content is gained by means of a layered or nested menu navigation system which due to the program structure can be easily extended to accommodate changes and additions. A significant feature of developing with SCALA has been the substantial time saving over the more widely used authoring environments. Typically, in the author's experience, a months graphic and preproduction work can be programmed and completed in a day whereas normally, programming can easily take longer than the graphics production time. The logic of the programming functions substantially obviate debugging although the script generated by SCALA can be modified using a standard ASCII text editor.
Other key personnel involved in the project development include historian Dr Neville Green who has contributed his substantial skills in historical research and script development, multimedia integration consultant Anthony Temple and the Director of the WA Department of Aboriginal Sites Dr Nick Green and his staff. The author is also particularly indebted to the members of the Aboriginal community - performers, artists and musicians without whose contribution the production would certainly lack its cultural richness.
The paper will be illustrated with a multimedia presentation.
| Author: George Borzyskowski Senior Lecturer, School of Design Curtin University of Technology GPO BOX U 1987, Perth 6001 WA Tel: 09 351 2263 Fax: 09 351 2980 Please cite as: Borzyskowski, G. (1994). Dinosaurs, gas and Alec. In C. McBeath and R. Atkinson (Eds), Proceedings of the Second International Interactive Multimedia Symposium, 32-37. Perth, Western Australia, 23-28 January. Promaco Conventions. http://www.aset.org.au/confs/iims/1994/bc/borzyskowski1.html |