IIMS 1994: Quinn - implications of instructional and game design for interactive multimedia interfaces

Implications of instructional and game design for interactive multimedia interfaces

Clark N Quinn
The University of New South Wales

Instructional programs and computer games are two sources of constraints on multimedia system interface design. Many of the best and most innovative interface designs have arisen from computer games. Advances such as windows, rooms, and direct manipulation were first seen commercially in programs such as FaceMaker, Rocky's Boots and Pinball Construction Set, respectively. Current computer games similarly exhibit information concepts that can guide interface design. Note that careful attention must be paid to the role that deliberate interface difficulty may have in making games challenging. Another source of interface implications is instructional systems. In most interfaces, the task is known to the user and can serve as information to bridge the gap between system and task. In instructional systems, however, the user not only has an interface imposed on the task, but the task itself is learning new information. Thus, the interface must be as transparent as possible, or to consider another approach, the interface actions must be the learning activities. In this paper I sketch some computer game and some instructional program interfaces and draw implications on making interface activities correspond to user goals.

Introduction

The link between our capacity to think and those objects we have designed to facilitate our ability to understand the world is the interface between user and system. As both our ability to reflect upon our own capacity for thought and the technological capacities we can wield grows, we need to look for sources of insight into how to bridge this gap most effectively. Currently we are linking multiple media of information with interactive design to suit multiple purposes including instruction, entertainment, and information. While the results of pure research on thinking, the various fields that comprise cognitive science, provide principles and guidelines for design, we must also look to those fields where empirical results have proven effective. Laurel (1991) has looked to drama for inspiration, while Tognazzini (1993) has looked at stage magicians for insight. I propose two further fields for insight into effective design: computer games and effective instructional programs.

A framework

To that end I want to introduce a particular framework for viewing the cycle of interaction with a system, Norman's (1988) Seven Stages of Action Model (See Figure 1). In this model of interaction, we start at the top with the user's goals, and work our way through a specification from intentions to executed actions. This path is known as the gulf of execution. A similar path, the gulf of evaluation, moves from perception to evaluation of the outcome. To design a useable system is to bridge these gulfs. Perception in this model includes the senses of vision and audition, while our actions typically consist of hand motions, either at the keyboard or with a pointing device (such as a mouse or trackball, but the following discussion will be largely independent of this level of consideration).

Figure 1: Norman's Seven Stages of Action

This framework gives us an opportunity to talk about the various issues that arise in using a system. The input actions are accommodated through the gulf of execution, and the information presented through the various media are processed along the gulf of evaluation. The emphasis on the goals highlights the importance of designing for use. The gulfs also provide a way to talk about the "directness" (eg, Ziegler & Fähnrich, 1988) of an interface. An interface is direct to the extent that the mapping between goals and actions is straight forward. Two components that affect this mapping are semantic directness and articulatory directness (Hutchins, Hollan, & Norman, 1986). Semantic directness arises from the relationship between the goals and the meaning of the actions; a button shaped like an arrow to indicate direction of movement is direct, while an arbitrary button shape is less so. Articulatory directness arises from the relationship between the action taken and the physical instantiation; turning a knob to rotate an object is direct, using a slider would not be. This can get confounded, of course, by system constraints: rotating screen displayed knobs with most pointing devices is unnaturally difficult as there is not the natural use of arm joint rotation that makes real knobs so effective.

Using computer games as inspiration for interface design is not new. Carroll (1982) compared players of the game Adventure with users of standard computer applications. More recently, Neal (1990) summarised several computer based arcade and simple mathematics games and their implications. Further, several of the most important developments in interface design were first popularised through computer games; developments like overlapping windows, direct manipulation, and room metaphors were first exhibited in FaceMaker, Pinball Construction Set, and Rocky's Boots, respectively. This suggests that this source of information should be extended to multimedia interfaces as well.

Games differ from other activities in particular ways. While the word game is typically used to illustrate how some concepts elude attempts to find defining features, Malone (1981) identified three components in computer games that contributed to enjoyment: challenge, fantasy, and, curiosity. Curiosity is the element of chance, not having a totally deterministic outcome. This can occur through random events, and through the process of discovery. Fantasy is, essentially, the compellingness of the story. Challenge is the requirement that games must maintain interest over time. For example, in games things are often deliberately made difficult to prevent trivial solutions. Game design includes evaluating trade offs between difficulty and a compelling and engaging story. The lack of directness in an interface may be a device to make the game challenging when an otherwise engaging plot and action match do not yield sufficient challenge alone.

Laurel's (1991) examination of the elements of drama that have implications for interface design provides a concept that captures important elements that contribute to fantasy, the concept of "engagement". One way to view the elements of engagement is to consider an engrossing game as having the elements of a good story or play (although Laurel discriminates between them, for the purposes here they can be considered equivalent). A compelling story is one that has coherency, tension, relevance, and closure. Similarly, a game should lead you reliably over obstacles in pursuit of a goal. These obstacles need to be more than trivial, yet not be arbitrary, and the conclusion of the game should resolve the events that occurred throughout the game.

There are a variety of different types of games to consider. Arcade games, fantasy role playing games, and adventure games are some of the traditional categories of games. Arcade style games are the fast paced, motor control testing graphic games that started with Pong, and have been seen in game arcades and characterise the home entertainment systems now so commercially popular. Fantasy role playing games are computerised equivalents of the popular D&D, or dungeons and dragons, games where parties of adventurers descend into mazes populated with monsters to be battled in a quest for treasure. Adventure games are single character explorations of simulated worlds where the correct use of found objects enables further exploration to some goal. Other popular categories of games are simulations of combat, or other adversarial situations including criminal investigations, corporate negotiations, and more idiosyncratic choices. However, as games get more sophisticated these boundaries are blurring. It is better to attempt to characterise games in terms of the different dimensions along which they can vary.

Real versus non-real time

One of the most obvious distinctions is whether the game is effectively "real time", that is whether the game requires immediate reaction from the user or whether the user can pause to ponder the situation. We see arcade style games that compel players to get further in the game by repeatedly trying and failing until the solution path has been discovered. Another approach is that seen in adventure games, where the user can consider the problem and reflect on information previously gathered to choose an action before making a particular choice and trying it out. Some games are intermediate to the extent that the time component, while not wholly tied to movement, is only proportional to the passage of real time, not directly in correspondence. A simulation game might have regular passage of time, but a few minutes of game play might correspond to the passage of years in the game world. Another way in which games can be intermediary between the extremes are mixtures of the two approaches: games that may have quiet puzzle solving opportunities in the midst of quick response game play, or vice versa. In terms of the action framework, the distinction is the chronological link between perception and a subsequent execution.

Novel versus automatisation

Related to the above dimension is whether the problem to be solved, or the inherent difficulty in the game, is something to become well practiced for efficient play, or whether the problem, once solved, is not to be seen again and new problems are encountered. Some games, typically arcade style, repeat essentially the same features in similar formats. Other games are continually presenting new types of problems to be solved. Whether general problem solving ability is required or specific skills should be acquired is a way in which games differ. Intermediate positions are possible here, where the puzzles or problems are similar, or that you move through classes of similar puzzles. In terms of the framework, we might interpret this to mean whether the mapping from intention to execution requires a well practiced response before conscious intervention or whether cognitive reflection is to be used to help bridge the gap.

Role versus abstract

Another way in which games can vary is whether the player is taking a role in the game or whether the game is abstract. For instance, a player can be a character flying anything from a plane to a spaceship to a gigantic creature, or can be placing markers on a board or acting on some arbitrary geometric figures on an essentially meaningless task. For instance, some games are based on totally arbitrary rules, such as card games. The constraints are intrinsic to the game and unrelated semantically to anything in the world. More often, there are degrees of relationship to a real activity, where some aspect of the world is simplified to allow the game to focus on a specific issue or type of problem. The use of roles can tap into knowledge of the user about the situation to bridge gulfs of execution and evaluation, while abstract systems require learning on the part of the user or explicit representations of the abstract constraints. Roles also make the game meaningful and help support engagement.

Text versus graphic

While the trend is weakening, adventure games are still sometimes found that exist only in a textual mode. Increasingly, however, games are going from text based to graphic. This can be seen as a distinction between a linear interface and a spatial one. The corresponding interaction change is from keyboard commands to pointing device actions or from a linear device to spatial devices. Of course, certain games benefit from a combination of the two media of graphics and text, and similarly there are games that combine pointing and keyboard commands. As the arcade style games extend graphics to dynamic animation, the responses move from location to gesture. Linking the response type to the presentation medium provides a correspondence between the two sides of the cycle of action.

Dedicated versus versatile

Several ways have been found in games to help provide players with information about successful use of the system. Arcade games often have dedicated hardware that supports appropriate actions. Trackballs, joysticks, and buttons provide unambiguous clues about how they can be used. Mapping such games to more versatile systems, like general purpose computers often requires somewhat arbitrary mappings, although allowing greater flexibility. One solution used in arcades is having an attract feature that explains the game while no one is playing. This feature has been expanded in some games for general purpose systems to the point of providing tutorials that can help users practice important skills.

Linear versus branching

Games can be linear or non-linear not just in medium but in structure. Linear games provide a single solution path and require the correct sequence of performance of particular skills or responses to situations, while non-linear games can allow the solution of puzzles in any order or alternate solutions to the same problem. For example, certain arcade games have a single correct sequence of problem solutions, while the best role playing games support different outcomes that are determined by player actions. The non-linear options place the player in more control, which can be empowering, but typically require more work in the design stage. On the other hand, linear games can be entertaining enough if the skill development is suitably challenging and the fantasy is sufficient to generate enthusiasm. However, from the point of view of the interface design, the more options available, the greater the decision about choosing an appropriate intention, while linear interfaces minimise flexibility in intention.

Instructional interfaces

The problems encountered in defining good instructional applications are similarly instructive. Instructional systems possess characteristics that make them another rich source of guidelines for design. The inherent nature of the learning task makes instructional environments more difficult to design for. The variety of learning outcomes also has relations to the tasks people use computers for. Just as there are dimensions along which games differ, there are dimensions along which instructional interfaces differ.

Known task versus new information

In one particular way, instructional interfaces provide a greater challenge than other interface design tasks. in interfaces for specific tasks, knowledge about the task can be used to bridge the gulfs. Assuming the user already understands the task, knowledge about accomplishing the goal can provide support for inferring the appropriate action in the new system. For instructional systems, however, such knowledge is not available. While the task of learning may share some features across topics, the fact that the user is trying to learn something that is currently unknown suggests there is a gap in knowledge that the interface cannot count on. There is a greater demand for the interface to be transparent, or to facilitate the learning. A general assumption in learning interfaces is that the learner cannot be expected to be a sophisticated user of computer systems, placing a greater emphasis on providing support for system use. In terms of the cycle of action, the user's goals are not completely specified, and must be discovered through use of the system.

Objects = concepts: Actions = activities

One way to resolve this dilemma is to equate the desired learning activities with interface actions so that the user's intentions of learning have the most direct mapping to the interface actions to be executed. If we view the learner's task as consisting of knowledge application, or a skill, instead of the mere acquisition of knowledge, we can suggest that a skill requires knowledge manipulation. This requires that the interface objects correspond to learning concepts and that interface actions consist of manipulations of those concepts (Quinn, 1993).

Regular user versus moving on

On the other hand, it may pay to consider the frequency of use of the user. If the learner will be returning to the same system, say for some complex knowledge base, or will be using the same system to learn a variety of different skills, it can be worth a time investment to provide training on the system rather than expect a novice to be able to use the system. A typical trade off in system design is learnability versus power. A system that is easily learned or immediately useable is generally not powerful, while powerful systems typically have steep learning curves. However, a steep learning curve may be a worthwhile cost if the system is to be frequently or intensively used. In other words, it may be worthwhile to spend some time on training the mapping between intentions and actions to allow more complex relations rather than have obvious but weak choices.

Locus of control

Instructional systems can also differ on the degree of user control the system exhibits. This is similar to the issue of linearity in game structure. Some learning systems direct you through a specific learning sequence, while others allow exploration by the user. For example, an EKG tutorial might specifically guide you through the key components of the heartbeat and abnormalities, while a circuitry environment might allow you to construct a variety of circuits and measure important parameters to discover the rules that govern such circuits. One of the important findings is that students do not reliably explore the full scope of explorable systems, but deplore totally guided or didactic systems. In terms of the framework, their understanding of the system, the goals of exploration are not supported by the system. One solution that has been seen is to provide a conceptual map of the system to be explored, so that there is an explicit representation of the scope of the system and the learner can then form a goal of exploring the entire space. It is also important that the system support recognition of which parts have already been explored.

Adaptivity

A related issue is the degree to which the system adapts to the user. Systems can have quite. sophisticated responses to learner inputs, or simply baulk until the correct answer is input. One answer to the issue above of learner control is to find a middle ground between an exploratory environment and a controlled one, where the system supports exploration by providing hints of areas still to be examined. This direction is best characterised by work in intelligent discovery environments (eg, Otsuki, 1993).

Design implications

The elements that characterise game and instructional interactions have implications for the design of new systems. The extension of the general solutions to the problems above to multimedia systems can lead to principles for design. We will refer to the framework to exploit those implications.

Starting at the top, the user's goals, it was seen that for learning systems, the space of goals that the system supported was not necessarily completely obvious. While intelligent support could provide support, such development is typically costly. While intelligent multimedia is an important area of research, such efforts are not always practical. Another way to support effective understanding of a system's scope is to provide an explicit representation of the scope of a system and one's current location. Particularly for pedagogical uses, such an overview helps support the user in creating particular goals for system use.

For frequent useability of the system, the trade off for powerful systems might argue for expanding beyond a simple interface. That is, moving from intention to execution could come from memory rather than inference from the interface presented objects. To support this, training would be required, but either the attract mode of arcade games or built in tutorials could provide a solution.

To bridge the gap of semantic directness, the example from learning systems may be applied. Making the interface objects reflect the information concepts and making the interface actions correspond to information manipulations can make information processing and instructional applications of multimedia systems effective. This mapping requires a careful analysis of the information representations and operations to the interface objects to minimise ambiguity. The goal is to make the perceptions provided by the system support the creation of appropriate intentions for execution.

To help the user map from intentions to actions, bridging articulatory directness, media specific responses were discussed. From games, actions appropriate to the form were used. As multimedia begin to incorporate dynamic representations, appropriate forms of interaction need to similarly evolve. However, gestural interfaces are still in their infancy (eg, Buxton, 1987). As spatial interfaces move to three dimensional representation, more complex interaction gestures will be needed.

The challenge element that contributes to game enjoyment by providing difficulties in choosing appropriate intentions or making execution difficulty may not be desirable for multimedia design. Multimedia aimed at providing information or instruction, as opposed to entertainment, may not benefit from the inclusion of interface difficulty for the sake of challenge. New learning or task accomplishment typically provides enough challenge. Similarly, curiosity from learning or information processing typically is more than sufficient and random events are to be avoided. Indeed, interface transparency as a goal in instructional systems is also a consideration in task oriented and learning oriented multimedia systems.

One particular way of increasing difficulty found in games should be avoided with one exception. The chronological time pressures that characterise arcade style games have a practical equivalent in training for and controlling of process monitoring. For systems where automating process control are not feasible or desirable (flying planes, for example), maintaining time pressure may be important. Training of process control, in particular, may be a very important application of multimedia technology. This relates as well to the issue of the role of the user. To engage the learners interest, the role playing can be motivating. To support learning the task may be simplified as in games, and then gradually have complexity added.

Engagement, however, is to be encouraged. While sufficient tension will come from the learning or task oriented aspects of use, the appeal of a system can be enhanced by the criteria that contribute to compelling game design: coherency and relevance. For example, aesthetic style is an element of graphic design and can be applied to interfaces. A coherent artistic appearance is more appealing, all else being equal. A semantic link between the artistic atmosphere and the intended nature of the system likewise increase the appeal of a system. Design guides can provide support for this task (eg, Marcus, 1992).

Conclusion

We want to use the action cycle to guide our design of interactive multimedia systems. Bridging the gaps and working from the users goals are our goals, and some of the implications above help. The link between the information provided by the interface and successful use comes from a consideration of the way in which people actually accomplish tasks.

References

Buxton, W. A. S. (1987). There's more to interaction than meets the eye: Some issues in manual input. In R. M. Baecker & W. A. S. Buxton (Eds), Readings in human computer interaction. San Mateo, CA: Morgan Kaufman.

Carroll, J. M. (1982). The adventure of getting to know a computer. IEEE Computer, 15(11), 49-58.

Hutchins, E. L., Hollan, J. D., & Norman, D. A. (1986). Direct manipulation interfaces. In S. W. Draper & D. A. Norman (Eds), User centered system design: New perspectives on human computer interaction. Hillsdale, NJ: Lawrence Erlbaum. Associates.

Laurel, B. (1991). Computers as theatre. Reading, MA: Addison-Wesley.

Malone, T. W. (1981). Towards a theory of intrinsically motivating instruction. Cognitive Science, 5, 333-370.

Marcus, A. (1992). Graphic design for electronic documents and user interfaces. New York: ACM Press.

Neal, L. (1990). Implications of computer games for system design. In D. Diaper, D. Gilmore, G. Cockton, & B. Schackel (Eds), Human-computer interaction: Interact '90. Amsterdam: North Holland.

Norman, D. A. (1988). The psychology of everyday things. NY: Basic Books.

Otsuki, S. (1993). Intelligent environment for 93: The discovery learning. Proceedings of AI-ED 93: The World Conference on Artificial Intelligence in Education. Edinburgh, Scotland.

Quinn, C. N. (1993). Strategy detection in graphic interfaces. Proceedings of AI-ED 93: The World Conference on Artificial Intelligence in Education. Edinburgh, Scotland.

Tognazzini, B. (1993). Principles, techniques, and ethics of stage magic and their application to human interface design. Proceedings of the International Joint Conference on Human Computer Interaction. Amsterdam, April.

Ziegler, J. E. & Fähnrich, K. P. (1988). Direct manipulation. In M. Helander (Ed), Handbook of human-computer interaction. Amsterdam: Elsevier Science.

Author: Clark Quinn, School of Computer Science and Engineering, University of NSW, Kensington, NSW 2033. Tel: 02 697 4034 Fax: 02 313 7987 Email: cnquinn@cse.unsw.edu.au
Please cite as: Quinn, C. N. (1994). Implications of instructional and game design for interactive multimedia interfaces. In C. McBeath and R. Atkinson (Eds), Proceedings of the Second International Interactive Multimedia Symposium, 455-460. Perth, Western Australia, 23-28 January. Promaco Conventions. http://www.aset.org.au/confs/iims/1994/qz/quinn.html

[ IIMS 94 contents ] [ IIMS Main ] [ ASET home ]
This URL: http://www.aset.org.au/confs/iims/1994/qz/quinn.html
© 1994 Promaco Conventions. Reproduced by permission. Last revision: 15 Feb 2004. Editor: Roger Atkinson
Previous URL 18 Aug 2000 to 30 Sep 2002: http://cleo.murdoch.edu.au/gen/aset/confs/iims/94/qz/quinn.html