This paper describes human computer interface issues as they apply to interactive multimedia environments and discusses 'mental models' as a focus for research regarding user interfaces in interactive multimedia. The design of better user interfaces remains a major challenge to developers of interactive multimedia products, a challenge that may be met by mental models research. The concept of mental models is explained, measures of mental models are presented, and an on going research study at Edith Cowan University is described.
The purpose for doing multimedia research is that knowledge of how users represent systems will lead to a better understanding of what is needed to design more useable systems. In particular it is assumed that insights into user's cognitive processes when using interactive multimedia (IMM) products will result in user interface designs that are more efficient and easier to use. Current design principles are largely based on beliefs unsupported by research knowledge of the way users interact with IMM environments.
The literature about user interface design for IMM ranges from the artistic (cf, Laurel, 1990) to the technical (cf, Blattner & Dannenberg, 1992). Laurel (1990) defines user interface as a combination of "the physical properties of the interactors, the functions to be performed, and the balance of power and control" (p.xii). In IMM, the interactors are the human learner and the IMM system itself. According to Laurel, the primary concerns in user interface design are the intentions of the user. From Laurel's perspective, everything about the user interface should serve to empower the learner to accomplish the tasks that he/she has determined (eg, learn a new language, experience time travel, or find information needed to solve a problem). The user desires to focus exclusively on the learning and information seeking aspects of the experience of using IMM. Any mental effort that the user must invest in understanding the user interface of IMM is effort that cannot be focused on the content of the program.
In their more technically oriented volume, Blattner and Dannenberg (1992) refer to interface design in terms of media (eg, text, audio, and video) and modes of interacting with media (eg, pointing, speaking, and keying). With respect to media, Blattner and Dannenberg "are most concerned with how the information content of a media is conveyed" (p.xxv). With respect to modes, they "are usually more concerned with the dynamics of interaction" (p. xxv). Their focus on interface design is first, making the modes of interaction as obvious as possible so that they even become "transparent" or "second nature" to the user, and second, making the media as appropriate as possible to the nature of the content.
The user friendliness of software has also become of increasing concern to designers of interactive multimedia (IMM). This concern is well placed. Any confusion involved in using programs designed for education and training is especially detrimental. After all, IMM programs are designed to help people learn, not bewilder them. Learners often end up lacking the clear understanding of the structure and functions of IMM intended by designers.
This problem can be especially acute in multimedia and hypertext environments that allow learners to explore the content and activities in the program according to their own needs and interests (Heller, 1990; Jonassen & Grabinger, 1990). According to Shneiderman and Kearsley (1989), hypertext "is a database that has active cross references and allows the reader to 'jump' to other parts of the database as desired" (p. 3). Hypertext, hypermedia, and multimedia are all being designed and used for instructional purposes; in each case, the materials allow learners to navigate through the contents and activities via largely unrestrained exploration. However, there is a price for the freedom these types of systems permit. Users of hypertext materials may become confused and quickly lose track of what is going on, what they can do, and/or where they are located in the program. The resulting frustration prohibits learners from taking full advantage of the learning opportunities presented by these systems.
Many factors influence interactive behaviours, such as nature of computer interfaces and the individual differences learners bring to the human computer interaction process (Canter, Rivers, & Storrs, 1985). Different understandings of interfaces are also likely to influence user's navigational patterns (McAleese, 1989). Navigational patterns are most likely to vary in IMM programs designed to allow a learner to explore content and activities at his/her own volition. Although it seems to make sense that learners' understanding of the structure and functions of IMM will have an impact on their navigational behaviours as well as learning performance, there is little research to support this assertion. Insufficient or ineffective interaction with a program leads to confusion, getting lost, and/or partial use, non-use, or misuse of the information provided by the IMM, getting bored, and/or frustration. These and other problems result in less effective learning performance.
On the other hand, it may be that disorientation in IMM is not necessarily bad. Kahn and Landow (1993) present an analysis that indicates that some types of disorientation may be appropriate, even pleasurable, especially when the learner is engaged in dealing with complex, novel material. However, it is clear that stressful, unproductive disorientation results from a poorly designed interface. Instead of thinking that learning outcomes are an exclusive function of IMM or of the learners, designers should recognise that there are multiple contributors to learning outcomes: the learners themselves, instructional content and activities, and the interface between the learner and program.
The cognitive demands of human learning are complex and involve various variables which interact with learner characteristics such as aptitude, age, gender, ability, experiences, and cognitive styles (Ackerman, Sternberg, & Glaser, 1989). Learning is influenced by three dimensions of individual differences factors: (1) personological factors such as aptitude, ability, skill, prior knowledge, educational level, experiences, and learning styles; (2) affective factors such as, value, self efficacy, motivation, attitude, and anxiety; and (3) physiological factors such as eye hand coordination and visual acuity.
Learners acquire knowledge about screen displays and figure out how to use IMM through unique ways. Learners learn how to use programs by forming mental models consisting of a few uncomplicated, low level heuristics and by making inferences based on mismatches between their expectations and actual system reactions (Lewis, 1986). A shift away from mere concern with specification of mechanical aspects of interfaces and toward developing a better picture of how users understand the workings of computer programs is a new direction in the field of human computer interaction (Ackermann & Tauber, 1990; Kerr, 1987).
The limited capacity of working memory to hold only 5 to 9 chunks of information simultaneously (Miller, 1956) makes it difficult for users of complex structured IMM to reason when numerous cognitive load factors must be handled at once. Users feel overwhelmed by abundant menus, option commands, icons, windows, functions, sequence controls, etc (Jih, 1991). This complexity increases the cognitive load. It also increases the risks of confusion, especially when users confront multimedia materials which intentionally include many interactive options. The possibility of user disorientation is a major concern in hypertext materials that feature a network like, more flexible and complex structure (Edwards & Hardman, 1989; Kulin & Streitz, 1989; Tsai, 1988/89).
Users seek new information in ways that depend on and are limited by their current mental model and learning goals (Tweney, 1987). The existence and value of the mental models concept (Moray, 1987; Rasmussen, 1990) can be summarised by the premise that the quality of interaction within system operation depends upon the functionality of the mental models users have of the system. Mental models are the vehicle for understanding overall system configurations, their elements, and functional interrelationships.
The risk of learner disorientation in IMM is decreased by the quality of the mental model of the interactive program possessed by the learner. Jih (1991) found that students with a more complete mental model of the graphical user interface (GUI) of the Apple Macintosh computer were more effective users of an IMM program that adopted elements of the Macintosh GUI. Research on mental models can identify salient characteristics of cognitive processes engaged in human computer interaction and help in the development of research based guidelines for the design of effective IMM. There is a need to identify the user's understanding about interfaces in term of mental models to provide principles and guidelines for design of IMM.
A mental model is a model evolving in the mind of a user as the user is learning and interacting with a computer system (Gentner & Stevens, 1983; Norman, 1983). The mental model represents the structure and internal relationships of a system, although in a simpler form. The user's mental model is the source of the user's expectations about the effects of actions, can guide navigation or planning of actions, and contributes in interpretation of feedback (Van Der Veer, 1989). It seems to make sense that an ideal mental model would be consistent with the conceptual model of the interface developed by designers, but this has not been demonstrated consistently. However, according to Norman (1983, 1986), strong or accurate mental models show a functional or spatial similarity to the system or to the image the system presents to the users.
Support for the existence and value of mental models in facilitating learning performance, retention of procedures, and the invention of new operational procedures can be found throughout the research literature (cf, Ackermann & Tauber, 1990; Jagcinski & Miller, 1978; Kieras & Bovair, 1984; Mayer, 1989; Moray, 1987). Mayer (1981) concluded that users systematically develop a cognitive structure or mental model for any task environment they engage. Individual differences relevant to the way novice users acquire a mental model are prior knowledge, information processing style, and general intelligence (Van Der Veer & Felt, 1988). Human factors research, indicates that mental models can help in users' interpretation of cues from the system, in working out appropriate actions, in detecting and repairing errors, and in learning (Green, 1990). Despite these claims, it is safe to say that the research on users' mental models is not yet well developed, especially with respect to the design and implementation of IMM. The lack of a strong theoretical basis for mental models is a continuing problem (Fetzer, 1993; Johnson-Laird & Byrne, 1991).
In the absence of strong theoretical support for design of IMM interfaces, two types of exploratory studies appear to be needed. First, qualitative analyses based on learners' descriptions of the interface and their explanations of their specific navigational choices can be used to categorise the properties of their mental models of IMM. For example, in an effort to understand what motivates exploration in IMM, Harmon (1992) asked users of a complex IMM program (ABC News Interactive Corporation, 1991) to "think aloud" during their exploration of the program. It is also appropriate to investigate the mental models of the designers of IMM programs. Jones (1993) interviewed designers of IMM programs to reveal the sources of the decisions they made in designing the user interface for their programs.
Second, if sufficient sample sizes can be reached, multiple regression methods can be used for gaining insights into the relationship among the learners' individual differences, their mental models of the structure of interface features in IMM, their actual navigation activities during use of IMM, and the results of that use.
There are at least four major questions which should be addressed in this line of research:
In the pilot study one class of pre-primary students (n = 25) and one split class of grade 1 students (n = 11) and grade 2 students (n= 17) were given access to the computer based version of the children's book Just Grandma and Me written by Mercer Mayer (1983). This program was chosen because of its multimedia nature and easy navigation interface. In each of the two classrooms, one computer was permanently ad up to run the program over a 3 week trial period. In both classrooms the teacher introduced the story by reading the book version and then rostered the children to have turns with the computer based version. In the pre-primary centre each child was assisted by a parent helper and in the split grade 112 classroom the children usually worked in pairs. When needed, a peer tutor (the class 'expert') was available to assist. All children had ample opportunity to use the mouse (the keyboard is not required for this program) as they interacted with the story. All children had at least 45 minutes exposure to the program.
After the three week trial period each child was invited to demonstrate various aspects of the program and to answer a list of 18 set questions posed by a research assistant (see Appendix A). This 'demonstration and interview' session took approximately 25 minutes per student. All sessions were videotaped for subsequent analysis.
The interview questions posed were designed to explore the mental model that each child had formulated about how to use the program and what the program meant to them. The interview process is described in Appendix A. Some questions related to another electronic book ("Cinderella") in order to measure the degree to which various aspects of the electronic book experience with Just Grandma and Me transferred to a new but similar situation.
There are several methodological conditions that must be met for these studies. First, the learners should be involved in purposeful learning, driven by either intrinsic or extrinsic motivation. Volunteer subjects using IMM not directly related to their education needs are inappropriate. Second, the learners should spend many hours rather than a few minutes interacting with the IMM. Third, ideally, the population of learners should be diverse in age, education, and prior experiences with computers. Fourth, if multiple regression methods are to be used, sample sizes of learners must be large, ideally running into the hundreds, to meet the requirements of these analytical techniques.
IMM candidates for this research include interactive CD-ROM multimedia programs produced by companies such as ABC News Interactive, Apple Computer, Inc, Broderbund Software, the IBM Corporation, and Voyager Press. Many of these programs feature a windows, icon, mouse, and pull down menus (WIMP) interface. Users view a screen and respond to the programs via mouse or keyboard input. These programs feature an elaborate array of multimedia options including audio, video, graphics, animation, and text.
Specific procedures appropriate to studies of mental models and their interrelationships with individual differences, navigational paths, and performance in IMM are:
Ackerman, P. L., Sternberg, R. L, & Glaser, R. (Eds). (1989). Learning and individual differences: Advances in theory and research. New York, W. H. Freeman.
Ackermann, D. & Tauber, M. J. (Ed). (1990). Mental models and human-computer interaction 1. The Netherlands: Elsevier Science.
Blattner, M. M. & Dannenberg, R. B. (Eds) (1992). Multimedia interface design. New York: ACM Press.
Broderbund Software. (1991). Just Grandma and Me [Interactive multimedia program].
Canter, D., Rivers, R. & Storrs, G. (1985). Characterising user navigation through complex data structures. Behaviour and Information Technology, 4(2), 93-102.
Edwards, D. M. & Hardman, L. (1989). Lost in hyperspace: Cognitive mapping and navigation in a hypertext environment. In R. McAleese (Ed), Hypertext: Theory into practice. Norwood, NJ: Ablex Publishing Corporation.
Fetzer, J. H. (1993). The argument for mental models is unsound. Behavioral and Brain Sciences, 16(2), 347-348.
Gardiner, M. M. & Christie, B. (Eds). (1987). Applying cognitive psychology to user interface design. New York, NY: John Wiley & Sons Ltd.
Gardner, H. (1983). Frames of mind: The theory of multiple intelligences. New York: Basic Books.
Gentner, D. & Stevens, A. L. (1983). Mental models. Hilldale, NJ: Lawrence Erlbaum Associates Inc.
Green, T. R. G. (1990). Limited theories as a framework for human computer interaction. In D. Ackermann & M. J. Tauber (Eds), Mental models and human-computer interaction 1. Human Factors in Information Technology No. 3. Amsterdam: Elsevier Science Publishers B. V.
Harmon, S. W. (1992). On the nature of exploratory behavior in hypermedia environments: Considerations of learner use patterns of hypermedia environments for the design of hypermedia instructional systems. Unpublished doctoral dissertation, The University of Georgia.
Hassett, M. R. (1990). Adding research tools to computer based video instruction programs. In D. W. Dalton (Ed), Proceedings of the 32nd Annual International Conference of the Association for the Development of Computer Based Instructional Systems. San Diego, CA, October 29 - November 1, 1990.
Heller, R. S. (1990). The role of hypermedia in education: A look at the research issues. Journal of Research on Computing in Education, 22(4), 431-441.
International Business Machines Corporation. (1991a). Columbus: Encounter, discovery and beyond [Interactive multimedia program]. Atlanta, GA: IBM Corporation.
International Business Machines Corporation. (1991b). Illuminated books and manuscripts [Interactive multimedia program]. Atlanta, GA: IBM Corporation.
Jagcinski, R. L, & Miller, R. A. (1978). Describing the human operator's internal model of a dynamic system. Human Factors, 20(4), 425-433.
Jih, H. J. (1991). The relationship among the structure of interfaces, users' mental models, and performance in computer based interactive courseware. Unpublished doctoral dissertation, The University of Georgia.
Jih, H. J. & Reeves, T. C. (1992). Mental models: A research focus for interactive learning systems. Educational Technology Research and Development, 40(3), 39-53.
Johnson-Laird, P. N. (1983). Mental models: Toward a cognitive science of language, influence, and consciousness. Cambridge: Cambridge University Press.
Johnson-Laird, P. N. & Byrne, R. M. (1991). Deduction. Hillsdale, NJ: Lawrence Erlbaum.
Jonassen, D. H. & Grabinger, S. R. (1990). Problems and issues in designing hypertext/hypermedia for learning. In D. H. Jonassen & H. Mandl (Eds), Designing hypermedia for learning, 3-26. Berlin: Springer-Verlag.
Jones, M. G. (1993). Guidelines for screen design and interface design in computer based learning environments. Unpublished doctoral dissertation, The University of Georgia.
Kahn, P. & Landow, G. P. (1993). The pleasures of possibility: What is disorientation in hypertext? Journal of Computing in Higher Education, 4(2), 57-78.
Kerr, S. T. (1987). Finding one's way in electronic space: The relative importance of navigational cues and mental models. Paper presented at the Annual Convention of the Association for Educational Communications and Technology. Atlanta, GA, February 26 - March 1, 1987.
Kieras, D. E. & Bovair, S. (1984). The role of a mental model in learning to operate a device. Cognitive Science, 8(3), 255-273.
Koubeck, R. L, Salvendy, G. & Eberts, R. (1987). Eliciting knowledge for software development. Behaviour and Information Technology, 6(4), 427-440.
Kuhn, E. & Streitz, N. A. (1989). Do users know how to act when menus are complex? A numerical model of user interface complexity. In F. Klix et al (Eds), Man-computer interaction research: MACINTER-II. North-Holland: Elsevier Science Publishers B. V.
Laurel, B. (Ed) (1990). The art of human-computer interface design. Reading, MA: Addison-Wesley.
Lewis, C. (1986b). A model of mental model construction. In M. Mantei and P. Orbeton (Eds), Human factors in computing systems: CHI '86 Proceedings. New York: Association for Computing Machinery.
Mayer, M. (1983). Just Grandma and Me. Western Publishing, New York.
Mayer, R. E. (1989). Models for understanding. Review of Educational Research, 59(1), 43-46.
Mayer, R. E. (1981). The psychology of how novices learn computer programming. ACM Computing Surveys, 13, 121-141.
Mayhew, D. J. (1992). Principles and guidelines in software user interface design. Englewood Cliffs, NJ: Prentice- Hall.
McAleese, R. (1989). Navigation and browsing in hypertext. In R. McAleese (Ed.), Hypertext: Theory into practice. Norwood, NJ: Ablex Publishing Corporation.
Miller, G. A. (1956). The magical number seven, plus or minus two: Some limits on our capability for processing information. Psychological Review, 63, 81-97.
Moray, N. (1987). Intelligent aids, mental models, and the theory of machines. International Journal of Man-Machine Studies, 27, 619-629.
Norman, D. A. (1986). Cognitive engineering. In D. A. Norman & S. W. Draper (Eds), User centered system design: New perspectives on human-computer interaction. Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.
Norman, D. A. (1983). Some observations on mental models. In D. Gentner & A. L. Stevens (Eds), Mental models. Hillsdale, NJ: Lawrence Erlbaum Associates Inc.
Peterson, P. L. (1982). Individual differences. In H. E. Mitzel (Eds), Encyclopedia of educational research, 844-851. New York: The Free Press.
Price, R. V. (1991). Computer aided instruction: A guide for authors. Belmont, CA: Wadsworth, Inc.
Rasmussen, J. (1990). Mental models and the control of action in complex environments. In D. Ackermann & M. J. Tauber (Eds), Mental models and human computer interaction 1. Human Factors in Information Technology No. 3. Amsterdam: Elsevier Science Publishers B. V.
Reeves, T. C. (1993). Pseudoscience in computer based instruction: The case of learner control research. Journal of Computer Based Instruction, 20(2), 39-46.
Rogers, Y., Rutherford, A. & Bibby, P. A. (Eds) (1992). Models in the mind: Theory, perspective and application. San Diego, CA: Academic Press.
Shneiderman, B. (1987). Designing the user interface: Strategies for effective human- computer interaction. Reading: Addison-Wesley.
Shneiderman, B. (1982). The future of interactive systems and the emergence of direct manipulation. Behavior and Information Technology, 3, 237-256.
Shneiderman, B. & Kearsley, G. (1989). Hypertext hands on: An introduction to a new way of organising and accessing information. Reading, MA: Addison-Wesley Publishing Company, Inc.
Sternberg, R. J. (1985). Beyond IQ: A triarchic theory of human intelligence. Cambridge, MA: Cambridge University Press.
Tsai, C. J. (1988/1989). Hypertext: Technology, applications, and research issues. Journal of Educational Technology Systems, 17(1), 3-14.
Tuckman, B. W. (1988). Conducting educational research (3rd ed). Orlando, FL: Harcourt Brace Jovanovich, Inc.
Tweney, R. D. (1987). What is scientific thinking? Paper presented at the Annual Meeting of the American Educational Research Association. Washington DC, April 23-25, 1987.
Van der Veer, G. C. (1989). Individual differences and the user interface. Ergonomics, 32(11), 1431-1449.
Van der Veer, G. C. & Felt, A. M. (1988). Development of mental models of an office system: A field study on an introductory course. In G. C. van der Veer & G. Mulder (Eds), Human-computer interaction: Psychonomic aspects. Berlin Heidelberg, Germany: Springer-Verlag.
|Authors: Geoff Ring, Chairperson, Department of Computer Education, Edith Cowan University, Mount Lawley Campus, 2 Bradford Street, Mount Lawley WA 6050. Tel. + 61 8 9370 6369 Fax. +61 8 9370 2910. Email: firstname.lastname@example.org|
Rod Ellis, Edith Cowan University
Dr Thomas C. Reeves, The University of Georgia
Please cite as: Ring, G., Ellis, R. and Reeves, T. C. (1994). Mental models research and human computer interface design. In C. McBeath and R. Atkinson (Eds), Proceedings of the Second International Interactive Multimedia Symposium, 485-493. Perth, Western Australia, 23-28 January. Promaco Conventions. http://www.aset.org.au/confs/iims/1994/qz/ring2.html