|
My aim in presenting this paper is to share some thoughts, concerns and ideas about interactive multimedia from the point of view of one who has been involved for over 20 years as a graphic designer working across a variety of media forms. These have included print, television, exhibitions, instructional, documentary and animated film, where my responsibility has been for the practical and creative aspects of re-presenting information visually, both pictorially and typographically, in ways which make it more accessible to the understanding and comprehension of its end users, and through this intervention as a graphic designer, contributing a degree of efficiency to the communication process which occurs.
The graphic designer's function is that of an interpreter or intermediary between the source and the receiver of information. The basis and usefulness of the input which is brought to this process lies in the designer's ability to bring a range of skills and deductive processes to bear in manipulating elements, particularly visual elements, within the commonality of experience which exists between on the one hand the originator of the communication and on the other its intended recipient or audience. To put it another way, through the creation or adaptation and configuration of what amount to sets of commonly recognised visual signals, to attempt to elicit correspondence between the internal meanings of both sender and receiver resulting from a given communication transaction. This is applicable irrespective of ultimate objective whether it be entertainment, education, information or calling forth to action.
It may be worth reminding ourselves at this point of the much used and here adapted basic model of communication first described in 1949 by Shannon and Weaver, subsequently developed by Schramm and others. (Shannon & Weaver, 1949), (Schramm, 1954), (DeFleur, 1972). Although the interpretation used here is neither original in concept or particularly sophisticated, it attempts to indicate semiotic as well as mechanistic relationships within the process and can serve as a useful tool in order to get a better sense of where and how the input of the graphic designer fits in to the overall scheme of things and perhaps also remind us of the general context within which interactive multimedia is intended to function.
Figure 1: An adapted diagram of the communication process
illustrating the concept of commonality of experience
A distinction worth noting here is that which differentiates data from information. Data which can represent vital information within a given context to one person might well, if perceived at all, appear as simply more data to another, depending upon the frame of reference applied. Data is the raw material of information; the raw material of communication. In this representation the source derives information through the application of contextual criteria either consciously or subconsciously, to the selection and assessment of data available in any given situation in order to elicit meaning.
Within this distinction it can be seen that the evolution of computer technology is giving rise not so much to an information explosion but what can be more accurately described as a data explosion, (Mauro, 1985) hence the requirement of specialised tools in the form of software which allow us to manage sort and sift the data, into information from which we can extract meaning.
In the schematic model of communication there are represented a series of stages which are involved in achieving the objective of communication; that is, as similar an experience of meaning as possible by the receiver at the destination as exists for the sender at the source. (DeFleur, 1972). This requires articulation of meaning into symbols by the sender or encoding into a signal suitable for transmission via a channel or medium (Fiske, 1982). This is the point at which the artefacts or physical events of communication occur; ie. speech, morse code, graphic images, gestures, sounds, writing, music, films, television programs etc. These are produced according to the channel or medium selected for transmission, which in terms of human communication might be air through which sound waves travel, a book which carries the printed symbols of language, television which delivers audio visual signals or any of the range of other media with which we are familiar and have shared access to. Signals delivered by any of these may subsequently be received, decoded and interpreted into meaning and hopefully, correctly appreciated at the destination.
Cultural, psychological or mechanical influences can impair the process at various points. Shannon and Weaver identified extraneous mechanical influence within their model as noise. Typical sources of noise could be a crackly telephone line, out of focus projector or impaired hearing; but also can result from flawed articulation of the original meaning on the part of the sender, incorrect pronunciation of language; or poor encoding within the chosen medium, such as bad acting in a film, or contextually inappropriate style of representation in a drawing.
The effectiveness of the whole process relies primarily upon the degree to which the field of experience of the sender overlaps or is in common with the field of experience of the receiver (Schramm, 1954) given the medium or channel employed. This could be a language with which both are familiar and the degree to which the sender adheres to convention, protocol and the rules of syntax, paradigms applicable in ifs use. When differing paradigms form the basis of understanding of the sender and receiver, conflicts in interpretation occur. This example (Fig. 2) of packaging design resulted from a first year, student design exercise we call "text out of context". It clearly demonstrates the reliance we place upon culturally acquired paradigms.
Figure 2: Text out of context student exercise.
The amount of flexibility within a given coding system such as speech, or within a combination of systems to articulate a given meaning in different ways, yet still have the same meaning occur at the destination, is known as redundancy within the system (Fiske, 1982). It provides the possibility to communicate in spite of noise. It is the degree of predictability or possibility for metaphor, metonym and inference within a transaction of communication. For example the amount of variance which can exist in handwriting while still maintaining legibility and represent meaning. The fact that we are able to recognise traffic lights in a sketch in the absence of photographic detail implies redundancy within the processes of visual communication. Redundancy is vital to normal day to day human interaction.
Symbolic systems, on the other hand which are devised specifically to eliminate redundancy or room for interpretation are known as entropic (Fiske, 1982). Computer programming languages are like this. If a command is misspelt or used in the wrong way, the program will crash or a syntax error will occur.
Feedback or the ability for the receiver of communication to interact with the sender is the mechanism which completes this basic model, but rather than depicting feedback as a simple back looping arrow it is more useful when using the model as an analytical tool, to consider the feedback mechanism in the same detail as the primary communication mechanism. Feedback offers the receiver of communication the possibility to effect rate, quantity, quality of what is received in order to enable better decoding within the scope of redundancy characterising the particular system employed. The following current affairs item is an example of the process in action. The ongoing feedback within the conversation finally allows the "penny to drop" for the interviewer, that what had actually happened was that the house seller's pronunciation of "no agents", presumably over the telephone to the classified advertising department of a newspaper had subsequently appeared in print as "no Asians", hence the resulting cross purpose interview.
It is dear from a cursory examination of the model that the processes it seeks to describe, occur both multiply and simultaneously, at both micro and macro levels whenever communication takes place. The degree and possibility for feedback is dependant upon the channels available within the transaction and the prevailing circumstances.
Ultimately the whole process is about meaning, which as a cognitive function is not a physical entity capable of being actually moved as it were, from mind to mind or place to place. The production of similarity in cognitive function or meaning at both the source and destination is the purpose of communication through the movement of encoded signals or messages between them. (De Fleur, 1972).
Within the concept of "interactive multimedia", what we have is a further mechanical medium among those available for use in this process. The "new" medium provides the capability to incorporate a range of other existing media to the extent that they can be integrated and represented on a computer or video screen with sound, and allows the possibility for feedback in the form of mechanical control over the rate and sequence of signals received but within the constraint of general equipment capability and the sophistication with which the content has been organised by the programmer.
The possibilities of interactive multimedia were hinted at in 1963 in a paper entitled "A conceptual framework for the augmentation of man's intellect" by computer scientist Douglas Engelbart who at the 1968 Fall Joint Computer Conference in San Francisco demonstrated a graphical hypertext implementation which used an invention of his, the mouse, which moved a cursor around on the screen display allowing selection and expansion of keywords. The demonstration of Engelbart's "human augmentation technology" also included live video imagery in a window on the same screen. The achievement of this feat involved surmounting tremendous technical difficulties. Computer based interactive multimedia hadn't been done before. (Kay, 1987), (Time-Life, 1986).
Today the technological tools for both producing and delivering interactive multimedia are readily available, together with an increasingly clever range of software tools with which to organise, access, sequence and display the content. Developing and presenting the content is the difficulty.
Interactive multimedia has the capacity to incorporate existing media forms which in their own right have attained a level of maturity and sophistication of expression, maintained and evolved through the expertise of specialist practitioners and the aesthetic expectations of audiences. When the formal representation values within a medium are compromised, the difference between expectation and actuality, for the audience will inevitably detract attention from the content to the medium. If a photograph is scratched, the thing we are likely to pay a lot of attention to is the scratch. If an interactive application generates verbal responses with crude computer speech synthesis, we might be unnecessarily reminded of the computer and be distracted from our task. If an information system interaction menu is cluttered with unfamiliar, incomprehensible icons, we are likely to walk away. Our attention is drawn to the inconsistency and away from the expectation. The medium becomes the message (McLuhan, 1967), and the message of computers for the majority, still, is "too hard". These sorts inadequacies in expected formal representation within a given medium in terms of our communication model are identified as noise.
While interactive multimedia provides us with a unique means of engaging with information and simulating real experience, it also by virtue of multiplicity of media contexts, presents more opportunities for incongruity or noise than in individual media forms alone.
The graphic designer's responsibility working in multimedia production within a team situation, is to ensure contextual consistency of form in the visually expressed content, and to minimise as far as possible the incursion of the overt physical qualities of the medium, that is unless the application of medium specific paradigms is appropriate. This is not a question of merely applying a style or of decorating the content to look like something; but actually to determine its visual form in order for it to function in a particular way within a particular context.
The visual symbols and imagery used in communication generally possess connotations in addition to the meanings which they primarily represent. (Fiske, 1982). Basic symbols Sets such as typefaces evoke a wide variety culturally based effects beyond the immediate messages for which they are used. They are designed deliberately with that in mind, not merely for the sake of legibility or differentiation of emphasis. Inappropriate or insensitive selection of a typeface style though beautifully executed can and does result in the same sort of confused message as was shown in the "text out of context" example. The use of separately sourced imagery, dip art or clip media, when the visual style is not contextually valid is equally potentially disruptive of the communication intent. Similarly the particular arrangement and movement of text or other visual elements within the viewed space is a powerful determinant of the way in which information is read and decoded at the destination. Other considerations include the function of colour, texture and contrast, clarity and legibility, and ergonomics of use with regard to key reference elements and interaction access points within the screen layout. Questions of culturally or functionally based visual genre and the overall visual environment within which communication is to occur require careful and skilled attention.
Many of these issues are easily taken for granted and assume secondary importance as mechanical and logical functionality tends to dominate the development process, however if a meat pie doesn't taste like a meat pie is supposed to, in spite of containing what appear to be the right ingredients, the consumer is likely to spit it out. A crude analogy, but one which is borne out I am sure, by the experience of many of us when investigating computer games or simulation and other software. When the visual presentation looks amateurish and ill conceived or control functions are poorly depicted, the leap of credibility required to persist becomes too great to bother with the experience any further. "Good graphics" are the ingredient which provides the flavour of reality in a simulation and encouragement of positive confidence to engage in a learning experience in which the visual environment assists.
In enlisting the services of a graphic designer into a project it is important to establish consultation early enough in the development stages to allow the visual solutions required to evolve within a collaborative process of professional interaction. A final stage exercise to visually "fix up" an almost completed project in the absence of sufficient time for the designer to adequately appreciate the appropriate context for the visual elements required, is likely to end up looking "fixed up". Over reliance and indiscriminate use of some of the clever though hackneyed screen transition effects, wipes and fades often available in authoring software, to compensate for an otherwise indifferent visual feel, or to mask slow system response, can also be counter productive when perceived by end users as substandard versions of effects seen daily on television. While as multimedia developers and designers we are justifiably impressed with the increasing graphical and kinetic capability of the software and computer platforms we use, it is important to be mindful of the visual sophistication of the potential end users of interactive multimedia as a result of normal, daily exposure to existing media forms.
The ultimate value which good graphic design can contribute to an interactive multimedia production, after the normal, definable criteria such as visual consistency, appropriateness, clarity, etc, have been satisfied, is a quality, the actual aesthetically perceived physical manifestation of which is not empirically predictable. It is dependant upon the designers intuitive capacity to anticipate nuances of cultural change which constantly occur and through creative synthesis assert new paradigms of visual expression. The result shows up as a stimulating fresh and satisfying visual solution.
In conclusion, I would like to urge all those of us engaged in interactive multimedia in one form or another, whether within a commercial or an institutional environment, to pay some attention from time to time to the more vernacular and underground manifestations of this electronic communication mode. Within the files on local, national and international bulletin boards and increasingly in public domain software libraries, there is evidence of a vibrant, intercontinental, audio visually oriented youth subculture. Among the frequently interactive, animated audio visual "Demos" , "Intros", "disc-zines" and general electronic graffiti, are examples of astonishing virtuosity in software authoring, musicianship and visual artistry. Although much of this "non-professional" production is obviously motivated by a "Mine's bigger and better than your's" mentality, clearly the work is indicative of much that is engaging and audio visually exciting to a youth and adolescent audience. The aesthetic expectations and media sophistication of this audience, as indeed others, should not be overlooked when developing interactive multimedia applications which they will use.
Schramm, W. (1954). How communication works. In The Processes and Effects of Mass Communication. Urbana, University of Illinois Press.
DeFleur, M. L. (1972). Theories of Mass Communication. New York, David McKay Company, Inc.
Mauro, C. A. (1985). Coping with complexity. Paper published in Design in the Information Environment: How computing is changing the problems, processes and theories of design. New York, Alfred A. Knopf.
Fiske, J. (1982). Introduction to Communication Studies. London, Methuen & Co Ltd.
Kay, A. (1987). Doing with images makes symbols: Communicating with computers. Video recorded lecture. Stanford, University Video Communications.
Editors of Time-Life Books (1986). Understanding Computers: Input/output. Amsterdam, Time-Life Books.
McLuhan, M. and Fiore, Q. (1967). The Medium is the Message. Harmondsworth, Penguin Books Ltd.
| Please cite as: Borzyskowski, G. (1992). Interactive multimedia - a graphic designer's view, or Do you see what I mean? In Promaco Conventions (Ed.), Proceedings of the International Interactive Multimedia Symposium, 53-60. Perth, Western Australia, 27-31 January. Promaco Conventions. http://www.aset.org.au/confs/iims/1992/borzyskowski.html |