Full Paper

Martie Sanders and Ellis Ayayee

Martie@gecko.biol.wits.ac.za, Ellis@gecko.biol.wits.ac.za

School of Science Education,

University of the Witwatersrand,

South Africa

Educators and instructional designers are concerned that too often, in spite of careful instructional design, CAL programmes are not widely or effectively used by the target audience. Campbell strongly recommends that before designing instructional materials a "market research" survey of intended users is conducted to establish their motivation and needs, and Reeves (1994) suggests that accommodating individual differences should be a major factor in designing effective CAL programmes.

The success of CAL depends on many factors - hardware-related, software-related, and user-related. It is essential that all of these are considered when designing CAL programmes - an immense task (Kotze and de Villiers, 1996). This study set out to examine just one factor, which we saw as the first step in designing usable and used programmes, viz. students' aesthetic preferences. Grabinger (1989) claims that many design factors are based on folklore from the visual arts, and suggests much research still needs to be done. Many designers examine aesthetic preferences during the formative evaluation of software, only after it has been at least partially developed. We set out to investigate, before CAL materials were designed, learner profiles of the target audience so that the materials developed by our research and development team were more likely to be used because of their use of computer-appealing features or CAFs (Sultan and Jones, 1995). By doing this before we started developing the software we hope to avoid the clichÈd pitfall of putting the cart before the horse.

Grabinger (1989) emphasises that screen designs cannot be expected to make a major difference in the effectiveness of learning, and that we should look for only small effects. However, assuming that the instructional design of a programme is effective, he points out that screen design is very important if it can affect the motivation of the user to use the programme. Rimar (1996) emphasises the importance of "message design" for screen-based programmes (the manipulation of the physical form of the message) not just for aesthetic reasons, but because studies have shown that screen-based programmes with a good "message design" teach lessons more effectively.

Satisfying the majority of the students is particularly important when student groups are very heterogeneous, and when cultural beliefs, traditions and attitudes of the students may conflict with those of the instructional designers. Too often, through sheer ignorance, designers use inappropriate design components or expect practices contrary to those the students see as "good learning", thus bruising cultural sensibilities.

To meet the word limit for this paper we have removed the descriptions of the South African context and our university students - the context which provides the rationale for our study. An expanded version of this paper will be available at the conference, for those interested. Here we mention only that first-year classes in the biological sciences at our university are typically large, culturally diverse, and heterogeneous in terms of gender, home language, achievement levels, preparedness for university studies, and attitudes to learning. They are characterised by increasing numbers of educationally disadvantaged black students (see Figure 1), many of whom are considered to be educationally "at risk".

Although a variety of support mechanisms have been used since the late 1970s to help at-risk students adapt to university life and to cope with their academic studies, one approach not previously tried at the undergraduate level has been the use of Computer Aided Learning. CAL has a number of obvious advantages which suggest it is a logical approach to improving teaching and learning success in large classes which are heterogeneous in terms of student preparedness and performance. However, it is the affective advantages which are of particular importance to at-risk students, many of whom exhibit learning behaviours and negative attitudes which adversely affect meaningful learning (Sanders, 1986; Cummins, 1991; Grayson, 1995). As well as the technology being motivating, well-designed computer programmes allow for self-paced learning in a non-threatening environment - essential for students who often have low self-esteem and who tend to lack confidence. Amory and Mars (1994) have shown that the use of technology did not further marginalise their educationally disadvantaged students as might be feared with a student group relatively less experienced with computers.

We know that gender differences exist in computer experience, involvement and interaction with computers, and attitudes to using computers (e.g. Williams et al., 1993), and motivational aspects of learning (e.g. Huang et al., 1994), and should be considered when designing appropriate software. In addition, cultural differences amongst learners are strongly emphasised as a vital aspect in instructional design, yet are often ignored. Gardner (1994) believes failure to consider cultural aspects could lead to the failure of IT in developing countries, and Reeves (1994) lists "cultural sensitivity" as one of the ten dimensions in his Model of Effective Dimensions of Interactive Learning. Research has shown that this issue is particularly important in South Africa (Amory and Mars, 1994; Andrews, 1994; Fourie and Henning, 1994), where black students show a high dependence on the authority figure of the teacher (Grayson, 1995) and are more threatened than white or Indian students by the constructivist approach to learning (Amory and Mars, 1994).

With a research and development team comprising an award-winning botany "lecturer", and several experienced and successful science educationists, we felt fairly confident about being able to incorporate effective teaching and learning approaches in our software, but less sure about what our students would enjoy using - what would "turn them on" to learning. The literature abounds with reports on research about design variables which can be used to increase the motivation of learners so they become "mindfully engaged" (Salomon et al.,1991). A review is provided in our expanded paper, examining text density (Ross et al., 1988; Morrison et al., 1990; Rimar, 1996), font (Misanchuk, 1989; Rimar, 1996), colour (Grabinger, 1986; Rimar 1996), and the use of graphics (Rieber, 1991; Andrews, 1994; Sultan and Jones, 1995) and icons (Amory and Mars, 1994). However, because most studies seem to consider user groups as homogeneous (ignoring variables such as cultural background and gender), and because research results are often conflicting, we needed research-based answers before we started developing our software.

Our sample comprised 111 of the 128 students registered for the General Biology 1 course at the University of the Witwatersrand, South Africa. The group was made up of 42 black (22 female, 20 male), 23 Indian (17 female, 6 male), and 46 white students (30 female, 16 male), speaking 15 different home languages.

The context of the study (the need to develop appropriate - useful and used - software for the new undergraduate computer laboratory about to be opened) and its purpose (to find out about the users' backgrounds and preferences) were explained to the group by means of an animated computer-generated slide show which invited their assistance in the research. Phase One of the study elicited responses to realistic computer screens (as recommended by Morrison et al., 1989) shown by projection to the whole group, and their responses were collected by means of written opinionnaires. Phase Two of the research, which involves testing factors which require students to interact with the computers (e.g. timing, preferences for types of interaction, etc) has had to be delayed until 1998 because the new computer laboratory due to open earlier in 1997 is still not operational.

Where students were asked to indicate whether they "really liked", "liked", "disliked", or "really disliked" options they were shown, or whether they were "not sure", Approval Indices (AI) and Disapproval Indices (DI) with a maximum value of 1 were calculated as follows:

The maximum score was 2x the number of respondents.

As with all data based on people's perceptions, the results of this study should be interpreted with caution. Whilst some of the results were predictable, we found others quite unexpected. Many of them have important implications for how we should design our programmes, but some could well hold lessons for other designers. Two important trends are that a) cultural and gender differences were found in terms of students' prior computer experience, their attitudes to using computers, and their aesthetic preferences, and b) many of the students' choices conflicted with the preferences of the design team, choices which would have been used in the software developed.

The composition of the student group was extremely heterogenous in terms of race (37.8% black, 41.4% white, and 20.7% Indian) and gender (62% females, 38% males). These proportions are important when preferences of the different sub-groups differ. In any system of analysis which "vote-counts" users' views, the opinions of minority groups are likely to be obscured. However, it is important that they should not be ignored by CAL designers. A large proprtion of the group (38%) was "English Second Language" (ESL). Such students face additional difficulties when learning science, as even first-language students have trouble coping with the language of science (Sanders and Nhlapho, 1994; Moji and Grayson, 1997).

Only one student was colour-blind. Ferreira (1996) warns CAL designers who rely on colour to make information more understandable that colour vision deficiencies affect about 10% of the population, and we need to test a larger group of students to explain our unexpectedly low results.

We had expected few black students to have used computers at school, and to be relatively inexperienced in computer usage. Although we found this to be the case, many differences between the race groups were smaller than anticipated. Gender differences in computer experience were also found. Results will be reported more fully in the expanded paper.

We were encouraged that the majority of comments to our open-ended question asking how students felt about using computers were positive: 33 referred to the interest or enjoyment / fun value of computers, 17 to their value as teaching aids, and 8 to their helpfulness in accessing information. Nine students requested help on using computers, and six asked that they be made user-friendly. Figures 2 and 3 illustrate responses to a list of 28 descriptive terms to describe attitudes to using computers (see Sanders and Banda, 1995, for a discussion on the validity and reliability of such an instrument).

Several trends were noted.

• Students selected far more terms indicating positive attitudes, than words indicating negative attitudes.
• In terms of negative attitudes, males were more indifferent, and females more cautious and unsure. More females felt incapable.
• In terms of positive attitudes females seemed notably more interested and curious, and also more fortunate and privileged. Males were more interested and excited.
• Race differences noted in terms of negative feelings suggested the white students were more indifferent, cautious, uneasy and unsure, and felt more incapable than the black or Indian students, whilst the black students, generally, felt less negative than the Indian students.
• As noted in the study by Amory and Mars (1994) the Indian students seemed more positive than the white or black students in several instances. The white students were notably less excited, delighted and curious than the other two groups.

How prepared are students to use computer programmes teaching content and skills?

Electronic performance support systems (EPSSs) are more likely to be used if learners are computer inclined, and if they feel they need computer work (Barras-Baker,1994). Almost all students said they would use computers to learn about content and skills, although the white students would spend less time on it than the Indian or black students, particularly on programmes teaching skills important for academic success in biology (see Table 1).

The differences noted might be attributable to achievement level rather than race. The time high achievers (top third of the class) say they will spend on skills development is less than that of the low achievers (bottom third of the class) - possibly because the low achievers perceive more need to develop their skills. This is relevant to us as our software targets low achievers, and aims to develop a range of cognitive and metacognitive skills, as well as working on changing attitudes in order to influence work-related behaviours.

Three different types of background were tested: light, dark and textured. Table 2 illustrates, for the latter two types, how Approval and Disapproval Indices can help decide which features to use and which to avoid, when designing software. In order to satisfy the preferences of the maximum number of students, options with a high Approval Index and a low Disapproval Index should be used.

The following trends were noted:

• Although two of our three designers much preferred the dark screen backgrounds, students seemed to prefer the light-coloured screens to the dark ones.
• There were definite likes and dislikes for specific colours. The light maize background (although a preference of one designer) was disliked by most of the six race-gender sub-groups. Dark green (the strong favourite of another designer) was also unpopular, as were magenta and white.
• The favourite colour was blue, light blue being the most preferred option for the whole group, with dark blue second. Rimar (1996) recommends blue as a good colour for backgrounds, based on research he reviewed. Light green was the third most-popular option.
• Most students strongly disapproved of almost all the textured backgrounds (11 of the 19 Disapproval Indices were above 0.40, and the highest Approval Index for the whole group was 0.39. Most of the Approval Indices were below 0.20).

* Reported as a percentage this is misleading because of the small sample size.

We tested four options: normal graphs and diagrams, but no extra pictures; photographs; realistic drawings; and cartoons. The following trends were noted.

• Several gender differences occurred. Whist black females the realistic art-work (AI = 0.48) the black males did not (AI = 0.01). Amongst the white students a similar trend was noted for the photographs (AI females = 0.47, DI males = 0.65).
• All groups cartoons the best, particularly the black females (AI = 0.82) and white females (AI = 0.70). Although cartoons were the first choice of the white males they were not as approving as the other sub-groups (AI = 0.31). Our findings confirm those of Grabowski, reported by Sultan and Jones (1995), that student users like humour in CAL.
• Although the designers the use of photographs, many students, especially the males and the Indian females, disliked them (DI = 0.35 for black males, 0.33 for Indian males and females, and 0.65 for white males). AI scores were low except for white females (AI = 0.47). Most students gave as their reason that the photographs were not relevant to the screen (we thought they were).

We investigated seven different fonts, five of them sans serif (Arial, BinnerD, Comic Sans MS, Technical and Kaufmann Bd BT) and two with serifs (PT Barnum BT, Times New Roman). The results are summarised in Table 3.

• Contrary to the findings of Misanchuk (1989), Times New Roman was the most popular font with all the sub-groups except the black females (AI for the whole group = 0.57, DI = 0.05), closely followed by Arial (AI = 55, DI = 0.02). These preferences agreed with those of the designers.
• One font (Kaufmann) proved very unpopular with the students (AI = 0.16, DI = 0.43), although the designers all it. Although the font itself is quite attractive, text written in upper case is difficult to read.
• Two fairly similar modern fonts (Comic Sans and Technical) were popular with white and Indian students, epecially the females (AI = 0.53 for Indian and 0.57 for white females), but not with the black students (AI = 0.25 for black females and 0.23 for black males), or one of the black designers.

In terms of text colour, certain colours (notably yellow) were found difficult to read, although less so on dark backgrounds. Blue/green colour combinations were investigated to see whether, as rumoured, black students had trouble discriminating between blue and green (see Table 4). No race differences were apparent in the ease of reading dark green or blue text on a light green background. However, fewer black students than white and Indian found it easy to read dark green or dark blue text on a light blue background, or blue or green text on dark green and blue backgrounds. These trends need to be more carefully researched with a larger sample.

The results on screen density will be presented in the expanded paper.

We tested students' understanding of 16 icons. Their meanings were not always intuitively understood, although adding explanatory text to the icon improved comprehension.

Students were offered the choice of a number of alternative icons to perform particular functions, and asked to indicate their preferences. In almost all instances, the most preferred icons were those which included an explanatory text, as recommended by Amory and Mars (1994). The only exception was the icon with a question mark (indicating a route to "help"), where both genders of white students preferred the option without text.

Sixty-nine percent of the students were in favour of incorporating relevant sound effects, most citing enjoyment or motivational factors as the reason. However, five students felt sound would improve learning by aiding memory or improving concentration. Sixteen students felt sound would be distracting, whilst four advised the use of sound with headphones.

A number of the findings have implications for our software designers, and several findings could have a wider application.

1. A high proportion of English second-language students means careful language usage is essential.
1. Students are more computer literate than expected, although the black students are disadvantaged in comparison to the white and Indian students. Less basic training may be needed than we thought.
1. Students' attitudes to using computers for learning biology content and skills are more favourable than expected
1. Designers' preferences are not always a good indication of what students like.
1. Gender and race difference do exist for many of the computer-appealing features tested. Sultan and Jones (1995) recommend the use of CAFs to motivate learners to use computers. Software designers will have to take this into account when designing CAFs to motivate those students reluctant to use computers.

Our research suggests that if designers are developing materials on the assumption that their target audience is homogeneous, there must be sections of the group whose preferences are NOT considered and catered for.

(c) Martie Sanders and Ellis Ayayee

The author(s) assign to ASCILITE and educational and non-profit institutions a non-exclusive licence to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. The author(s) also grant a non-exclusive licence to ASCILITE to publish this document in full on the World Wide Web and on CD-ROM and in printed form with the ASCILITE 97 conference papers, and for the documents to be published on mirrors on the World Wide Web. Any other usage is prohibited without the express permission of the authors.