Full Paper

Don Cameron and David Treagust

d.cameron@info.curtin.edu.au

Curtin University of Technology

Concern over undergraduate occupational therapy students' capabilities and acceptance of the introduction of learning materials presented by electronic means were the reasons to undertake these studies. Curriculum changes were initially specific to an introductory unit in vocational rehabilitation, where students have the opportunity to take a more self-directed approach to their learning utilising information on the World Wide Web and CD-ROM. The radical changes from traditional teaching/learning methodologies, focusing on instructor presentation, to a more learner active role could disadvantage some students. For instance, students who have performed best in traditional learning environments may lose ground to colleagues with learning characteristics which are more appropriate to take advantages of learning through computer/telecommunications technologies (Gay, Trumbull & Mazur, 1991).

Many factors influence an individual learning in computerised interactive learning situations. These include motivation, previous experience, and a range of learner characteristics, including styles of functioning which impact on both the interpretation of processes and the appraisal of performance levels. Several kinds of styles have been identified by Messick (1994) including expressive styles, response styles, defensive styles, cognitive styles, and learning styles. Two of the most researched learner characteristics, which influence successful understanding in general, are cognitive style and learning style (Coventry, 1989; Logan, 1990; Sein & Bostrom, 1989; Van der Veer, 1989; Wood, Ford, Miller, Sobczyk, & Duffin, 1996). In addition, learning with information technology has proved to be influenced by users' computer experience and computer anxiety (Honeyman & White, 1987; Marcoulides, 1988; Nelson, Wiese & Cooper, 1991).

Users' performance browsing and problem-solving with multimedia learning materials is a comparative newcomer in the research literature, with limited examples of studies, particularly investigating links with learner characters. An associated area, which has many similar characters to multimedia learning, is computer bibliography searching which has an established history of over 30 years (Bellardo, 1984). Some areas where there is common potential problems for users with these two computer applications includes users' computer anxiety, navigating through materials where it may be difficult to visualise content structure and volume, and related problems with users' cognitive style and learning style. The study of bibliographic retrieval research is a promising and convenient starting point for an investigation into accessing information in a multimedia learning environment.

Several authors in the bibliography searching literature have referred to the 'puzzling' findings that appear repeatedly in studies which demonstrate lack of similarity by searchers in both process and outcome of on-line searches (Bellardo, 1985; Logan, 1990; Saracevic & Kentor, 1988). Despite numerous investigations, there has been a notable lack of definitive results identifying the characteristics associated with these differences. Fidel (1991) identified a drop in interest investigating on-line searching in recent years "because most experimenters have failed to provide conclusive results" (p.515). Fidel pointed out that although the term 'searching styles' is freely used in the literature in reference to on-line searching, there is no clear definition of what this term embodies. Some of the more promising results from the bibliography research have pointed to differences in user cognitive style and/or learning style as being a significant influencing variables in search performance (Wood, Ford, Miller, Sobczyk, & Duffin, 1996). However, others have found contrasting results showing no statistically significant relationship between these learner characteristics and search performance (Brindle, 1981).

Similar patterns of inconclusive results have been evident in the limited research linking multimedia navigation and performance with learner characteristics. Ellis, Ford, & Wood (1993) and Liu & Reed (1994) demonstrated that differences in field-independency/dependency cognitive style impact on the way individuals navigate, but that after the initial stages users representing each category performed equally well. In contrast, Repman, Rooze & Weller (1991) found that field-independent learners performed more effectively than field-dependent learners, both with and without the use of advance organisers. Melara (1996) measured participants learning style and found that Activist and Reflective users of hypertext were able to navigate hierarchical and network-like materials with equal success.

This exploratory study was designed to explore correlations between individual learning characteristics and navigation performance in two interactive multimedia learning environments. It endeavoured to investigate the formation of statistical models composed of these learning variables and their strength when carried over from one learning scenario to another.

All occupational therapy students studying an introductory occupational health unit in 1st semester 1996 at Curtin University volunteered to participate in these two studies. Two students were randomly selected to conduct pilot studies with a further sixty seven completing all sections. Students were offered a copy of the CD-ROM used in this study and a profile of their learner characteristics measured as inducements to participate in these two studies.

Both studies involved initially measuring each participant's learner characteristics. A range of measuring instruments/questionnaires were administered and included:

• Demographic data and technology experience questionnaire.
• Learning Styles Questionnaire (Honey & Mumford, 1992) to measure participant's ranking on four learning dimensions, i.e. Activist, Reflectors Theorist, and Pragmatist .
• Group Embedded Figures Test (Witkin, Oltman, Raskin & Karp, 1971) to measure field -dependency/field independency.
• Measuring Technophobia (Rosen & Weil, 1992) to measure participants ranking with three instruments: Computer Anxiety Scale, Computer Thoughts Survey, and the General Attitudes Towards Computers Scale.

The first experiment involved observing participants carry out four types of task utilising the CD-ROM Vocational Rehabilitation Learning Resource (Cameron, 1995). These tasks were:

• Carry out simple searches of a multimedia database
• Explore the package to become familiar with content and structure
• Conduct problem-solving exercises
• Teach-back the problem-solving features to the investigator

Students were permitted up to four minutes to complete each of the above group of tasks with instructions being given by the investigator who was located next to the participant in a small testing laboratory set-up. The participants actions were recorded by video camera and a research assistant located in an adjoining area monitored scores for a range of features (see Figure 1). Variables measured included: actual time required, screens accessed, successful searches, assistance requested and overviews accessed.

Following this section of the studies, participants were asked to complete two questionnaires assessing their knowledge of the CD-ROM content and their attitude to its design as an effective learning package. They then completed the second experiment which involved observations conducted when accessing relevant course information from the World Wide Web using the browsing software Netscape. Groups of tasks in this experiment were:

• Carry out simple searches of course related materials from the School of Occupational Therapy's Website
• Explore the World Wide Web to become familiar with content and structure
• Conduct problem-solving exercises with SafetyLine, Worksafe Western Australia's Website.
• Teach-back Netscape search procedures to the investigator

Similar time limits and observation procedures were adopted as for the first experiment and variables measured included: actual time required, screens accessed, successful searches, assistance requested and help features accessed. Following this section, participants were asked to complete two questionnaires assessing their knowledge of Netscape and their attitude to its design for ease of use.

Instrument reliability was measured for the learner characteristics instruments, frequencies measured, coefficient matrices constructed and multiple regression analysis conducted to identify if any of the learner characteristics predicted the performance outcomes.

Reliability analysis found that the General Attitudes Toward Computers Scale was unreliable, measuring alpha = .47 and was therefore not used in the experiments which followed. The results showed that a significant number of students (64%) had used computers at home to assist with their tertiary studies on six or more occasions in the past, but that only 21% had accessed the World Wide Web. Student's perception of their computer knowledge in comparison with their peers, identified that 96% considered they were average or below average. This response demonstrates either lack of confidence in this area or a lack of awareness of the ability of their peers.

The cognitive style dimension measured in this study showed there was a substantial number of therapy students (82%) who displayed field-independency characteristics. Field-independent individuals tend to be more analytical, impose their own structuring more on a situation, and be relatively less passive and global in their behaviour (Ford, Wood, & Walsh, 1994). Occupational therapists using information technology, and with a relatively high degree of field-independency, can expect to do better in tasks which require them to restructure information rather than accepting the structure provided by the materials (Jonassen & Wang, 1993). The Learning Style Questionnaire results showed that occupational therapy students have a preference for activist and reflector learning style. For instance, 48% of students displayed a strong to very strong preference for activist learning style, whereas 54% had a low to low preference for pragmatist learning style. With the technophobia instruments there was a significant number with 'no technophobia' i.e. 57% for computer anxiety and 41% for computer thoughts. However, a substantial number displayed low to high technophobia for each of these dimensions.

In the first experiment involving the multimedia problem-based learning package a correlation matrix displayed few significant relationships between the independent variables measured. Notable was the statistically significant correlations (p < .05) between the number of times that assistance was requested and cognitive style, computer experience, and computer thoughts. Backward elimination multiple regression analysis was employed to identify any subsets of variables that demonstrated prediction of the navigation performance dependent variables with the learner characteristics variables. Secondly, the attitude towards VRLR dependent variable was entered in a multiple regression analysis with the learner characteristics and navigation performance variables. (refer to Table 1).

The analysis identified statistically significant models for five of the six dependent navigation variables measured. Of the independent variables, age, cognitive style, computer thoughts, and prior computer experience all occurred in two or more of the statistically significant models for the navigation performance dependent variables. Attitudes towards VRLR was the dependent variable forming the strongest statistical model representing 38% of the variance.

With the Netscape experiment, regression analysis identified only three statistically significant models (refer to Table 2). Two of these were for the navigation performance dependent variables, time for searches and number of times assistance requested. Of the independent variable in this analysis; gender and pragmatist learning style were grouped together in a model partially explaining time for searchers. Computer knowledge, reflective learning style and computer thoughts partially explained number of times assistance requested. The dependent variable attitude towards the Web explained 31% of the predicted variance when coupled with the learner characteristics and navigation performance variables in a regression analysis.

This study was prompted by the planned introduction of interactive, problem-based learning into the curriculum at the School of Occupational Therapy, Curtin University. In a self-directed, interactive learning situation involving information technology, new users may be faced with a number of challenges as they navigate through the system and attempt to make its structure and contents meaningful. Not only must learners gain knowledge of new content matter, but they must also master the interactive technology.

As individuals attempt to make meaning from computer-learning systems they are likely to develop a model of its structure, content and function based on their previous experience and their cognitive processing characteristics. Developers of computer learning packages could benefit from knowledge of the intended users' characteristics that influence the development of models which lead to successful learning,. Thereby, they would be assisted in selection of features which could be incorporated to promote effective model formation.

Recent research has generally reported conflicting results from investigations into identification of statistical models linking learner characteristics, navigation strategies and attitudes in computer environments promoting learning. Previous studies in the areas of on-line literature searches and multimedia learning materials has observed a surprising amount of variation in the level of searcher performance, even with groups of the same amount of training and experience. Obviously, more research is required into influencing variables. This current research has identified moderate results, but still leaves substantial amounts of predictor variance not accounted for. At the same time, it has identified strengths and weaknesses of the learner population which should be of interest to multimedia developers and educators in general.

Evidence from this and previous research suggests that there is no single strong learner characteristic variable that stands out from the others. Other influences are at work including quality and meaningfulness of learning materials to users and the potential close association with student motivation. Two observations from this study are that there is an need to make students aware of their learner characteristics, and to cater for students who have an aversion to working with computer/telecommunications technology.

The former can be addressed, in the short term, by utilising existing learner characteristics instruments, such as used in this experiment. Knowledge of their strengths and weaknesses can be of benefit to students in self-paced learning environments. Measuring each student for these characteristics, making them aware of their strengths and weaknesses, then providing guidance on utilising their strengths and overcoming their areas of weakness can go some way to assist them in utilising this technology. However, there is also a need to develop new measuring instruments, which may incorporate some of the more relevant items from the instruments used in this study, but also measure other dimensions of personal styles which are assistive in accessing and utilising multimedia and Web technology. More studies must be undertaken to identify the coping strategies employed by successful users of multimedia and incorporating these in comprehensive instruments dedicated to measuring performance in this environment.

The significant number of students who demonstrated signs of aversion to using this technology in this study is of concern with increasing dependency of this media in today's learning environment. With the improved graphical interface of current generation computers and increased student computer awareness and computer literacy, the level of negative thoughts and anxiety was unexpected. On reflection, this aversion to this technology may be akin to the continuing dilemma many students face with anxiety when studying mathematics and sitting examinations. Overcoming this aversion to information technology should be a focus of educational research. Exploration of strategies of overcoming or minimising its effect may be assisted by employing cooperative learning environments, progressive introduction through rewarding experiences, and adequate help features to cater for divergent learners. Tertiary courses should be investigated to ensure that students' early experience with this technology does not kindle aversion to its future employment. In the past, occupational therapy students first formal exposure to computers in their undergraduate has been in a statistical unit. With many students traditionally struggling with this subject, without the potential threats from using computer technology, perhaps a more sensitive introduction to the use of information technology should be investigated.

The Web experiment was funded by a Telstra Social and Policy Research in Telecommunications Grant.

(c) Don Cameron and David Treagust

The author(s) assign to ASCILITE and educational and non-profit institutions a non-exclusive licence to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. The author(s) also grant a non-exclusive licence to ASCILITE to publish this document in full on the World Wide Web and on CD-ROM and in printed form with the ASCILITE 97 conference papers, and for the documents to be published on mirrors on the World Wide Web. Any other usage is prohibited without the express permission of the authors.