Full Paper

Scott Gazzard and James Dalziel

scottg@psych.su.oz.au, jamesd@psych.su.oz.au

Department of Psychology, The University of Sydney.

The use of computers, and more recently the Internet, has had a substantial impact on the nature of modern university teaching. The development of computer assisted learning packages by both universities and external groups (such as publishers) is one of the more important applications of computers within current education (Pittenger, 1997). By using computers to present course content, demonstrate phenomena, and to assess student knowledge, computers can help to assist teachers in providing learning resources for students that are not as labour intensive (in the long term) as small group teaching (Wolfe, 1995; Worthington, Welsh, Archer, Mindes, & Forsyth, 1996). While there are obviously certain aspects of small group teaching that cannot easily be supplanted by computer instruction, such as the skills that develop from sophisticated student-tutor dialogue, the computer can nonetheless provide an important role within the overall educational process.

However, many computer assisted learning packages have received limited use due to the difficulties of distribution and platform-dependence. Also, some packages have provided little or no opportunity for interactive learning, despite its value to student education (Entwistle & Ramsden, 1983) and/or have not made use of attractive presentation and contemporary appearance to encourage student engagement with content. In addition, there is still considerable need for evaluation of computer assisted learning materials, to determine student attitudes to this resource, and the helpful and unhelpful aspects of current practices.

Given the decreasing financial resources of modern universities, and the increasing sophistication of computer hardware, software, and networking, it is appropriate to consider the development of computer assisted learning packages as part of course development. In 1997, the Department of Psychology at The University of Sydney, as part of a revision of its introductory psychology program, developed a project to produce a computer-based tutorial on the World Wide Web (Web Based Tutorial - hereafter referred to as WBT) to present the topic of "Reasoning and Argument in Psychology".

This project was initiated with a number of aims, which can be separated into three categories: teaching and learning, software design, and evaluation. In terms of teaching and learning, the project was designed:

• To present material using a self-directed computer task, thus reducing the amount of preparation and intensive teaching required by tutors;
• To stimulate student learning via interactive exercises;
• To use an interesting, engaging multimedia format that incorporates examples from contemporary culture in order to make material more accessible and relevant to students; and
• To consolidate computer learning by applying new skills to video extracts that are also contemporary and accessible.

The second set of aims refers to issues of educational software design, specifically:

• To use the World Wide Web (WWW) so that material was platform-independent, available outside the University's computer labs, and available to external students; and
• To use the Java programming language for the implementation of truly interactive exercises whilst maintaining platform-independence.

The third set of aims refers to the goal of obtaining student feedback on their experience of using the WWW as a educational medium, in particular:

• To evaluate the tutorial via qualitative and quantitative questions; and
• To receive suggestions for the design of future similar software packages.

The WBT was utilised in the context of students' regular two-hour tutorial sessions. The first 90 minutes were devoted to the WBT, which presented and illustrated key concepts in scientific reasoning and argument. The final 30 minutes were spent in a tutor-led class discussion of several short video-clips chosen to give further examples of reasoning and argument. It was envisaged that the first session of the tutorial would introduce new concepts, and that the second session would consolidate that information.

In their completion of the WBT, students were guided by a set of notes from their student handbooks. These notes gave instructions about how and where to navigate through the tutorial and also explicitly required students to take notes on particular topics as they explored the information. This level of direction was introduced in order to maintain the students' focus on the task, and to highlight those pieces of information that were most important, as students may become anxious when encountering significant amounts of information without some structure to aid in the organisation and integration of that information. Students worked through this material in pairs, and were encouraged to collaborate and discuss material as they followed their handbook directions.

A short student questionnaire was created that collected quantitative and qualitative feedback from students. Questions covered the content of the tutorial, the format (i.e. the WBT), the overall quality of the tutorial, the perceived usefulness of the material, and students' enjoyment of the experience. Quantitative information was collected via student ratings of the tutorial on a 7-point scale with the following markers: 1-terrible, 2-very poor, 3-poor, 4-neutral, 5-good, 6-very good, 7-excellent. Students were also asked several open-ended questions to allow for qualitative feedback, such as: "What aspects of the tutorial could be changed to make it better?"

The tutorial was completed by approximately 1200 first-year psychology students. From this group, several classes were chosen at random to complete the questionnaire. This resulted in 146 respondents to the questionnaire. Completion of the questionnaire was voluntary and anonymous. Students were asked to respond to the tutorial overall, rather than just the WBT component.

On the whole, feedback on the tutorial was very positive. Students were asked to rate the overall quality of the tutorial, and their responses are presented in figure 1. The average response was 5.17, which is significantly above the neutral value of 4 ( t(145)=17.15, p<0.001 ).

When asked to nominate the most enjoyable aspect of the tutorial, students gave the following top responses (percentage of students giving each response in brackets): video clips (63%); working with computers (20%); interesting and relevant examples of concepts (16%); interesting subject matter (8%); interactive computer exercises (7%); and the self-directed nature of the task (5%). The most notable figure in these results is the overwhelming popularity of the video clips. This finding is unsurprising given that the WBT was a more challenging and intellectually demanding task, requiring students to learn a substantial number of new concepts, while the video clips were (often humourous) illustrations of those concepts, and called for much less effort. In summary, students appreciated the presentation of the material in a visually stimulating, interactive format.

Students did not report any major difficulties with navigation of the tutorial. When asked whether they experienced any problems, 67% of students reported none. Problems that were reported did not refer to difficulties in navigating content or using the computers per se but referred to environmental experiences. For example, the most common difficulties involved not having as much time as they would have liked to complete the task.

When students were asked how many tutorials of a similar format they would like to be included in their course (out of ten), the responses ranged from zero to ten, with a mean of 3.75. This data was positively skewed due to a notable group of students (12%) who preferred no similar tutorials. However, this group rated the tutorial equivalently to the overall sample (median of '5-good'), so it appears that their non-preference for similar tutorials stems not from a poor opinion of the quality of this particular WBT. Rather, they may simply favour a more conventional learning format in the classroom environment.

When students were asked about the least enjoyable aspect of the tutorial, several common themes emerged. The most popular response was nothing (20%), followed by tutorial length (18%), time restrictions (16%), note-taking during the WBT (14%), too much information (12%), and reading text from the computer screen (8%). Apart from the pleasing result that one-fifth of the students found nothing at all to complain about, this quantitative data taken in conjunction with other qualitative statements pointed to several time problems. The first problem was that the unbroken 90-minute computer session was too long for some students, who experienced fatigue, lack of concentration, and several cases of sore eyes. Many possible solutions to these problems were suggested by students themselves: less time on the computer (8%), less reading or note-taking (6%), splitting the WBT into several smaller components (5%), and shifting some information from computer screen to student notes (3%). The latter two suggestions are likely to be most appropriate for future revisions of the tutorial.

The second problem was, ironically, that students felt that the computer session was not long enough for them to thoroughly complete the directions provided by their handbooks. A common experience for students was that of feeling time pressure to finish the handbook note taking. The problem was simply that some students felt that there was too much material to cover within the time limits, which may have encouraged superficial note taking strategies at the expense of attempts to absorb and understand the material. Again, we may heed the advice of the students on this problem, and in future reduce the amount of note taking, break the tutorial into smaller components with rests in between, and possibly move some of the information out of the computer tutorial (at least for the purposes of the in-class session).

Unsurprisingly, some students did not enjoy the task of making notes on the textual content of the computer tutorial. However, the rationale for note taking, as given above, was that it is beneficial for maintaining focus on the task, and for aiding long-term retention of the material. It seems that at least some of the students appreciated this, with qualitative statements such as "note-taking not enjoyable, but necessary I think" supporting this view. Student difficulties with note taking should be alleviated if the above time problems are solved. It is also feasible that the amount of text presented to students via the computer screen in class could be reduced. Although studies show that reading comprehension for text presented via a computer display is not necessarily lower than equivalent texts presented on paper (e.g. Fish & Feldmann, 1987; Muter & Maurutto, 1991), students have in the past reported difficulties with absorbing large amounts of on-screen text. While this concern was kept in mind throughout the design of these pages and an effort was made not to present large unbroken sections of plain text, there is an unavoidable need here for the presentation of concepts which do require substantial amounts of reading. A solution that addresses all of the problems raised above would be to break the computer session into smaller components, and to intersperse these components with the presentation of information via other media, e.g. printed text, videos, tutor's instructions, and discussion.

A somewhat unexpected finding in the responses to questionnaires was that the social interaction provided within the classroom situation was a major factor contributing positively to the tutorial experience. For example, when asked to suggest changes for improvement to the tutorial, the most popular category of response (after 'nothing') was 'increasing social interaction' (16%) via group work, class discussions, tutor involvement etc. Aspects of social interaction were also common in response to the question of what was most enjoyable in the tutorial. Social interaction not only provides the obvious benefit of allowing knowledge to be shared among individuals, it also makes the learning environment more stimulating, and motivates students to complete the task. Some students also reported that the WBT provided an opportunity to meet others ("It's good to finally talk to someone at Uni"), and integrate further into the tutorial group.

The importance of social interaction points to an opportunity to utilise a potentially valuable aspect of the WBT experience in a classroom situation. As such, it also poses an interesting challenge from the point of view of software design: how does one structure a WBT that can successfully be used in two very different situations - as a learning aid in a class/group situation, and also as a purely self-directed individual information resource. The questionnaire results also highlight the differences between the two situations. In the first, 'individual user' situation, the student is browsing content at their own pace, is free to view content in any order they wish and can view as much or as little as they choose in a session. On the other hand, the individual user does not have the opportunity to ask for assistance or have the chance to discuss and clarify difficult material. In the second situation, the 'group-user' must conform to time and often resource restrictions, may be distracted by their environment, and will usually have less freedom to determine what they see, when, and for how long. They do however, have the advantages associated with their more stimulating and potentially informative social environment.

Consistent with what has been suggested above, we would suggest that the best strategy for achieving the dual aims of catering to individuals and groups is to take a component-based approach. The WBT can then be designed with separate components aimed at either or both types of user. Individual users would be free to 'pick and choose' the content that is most appropriate for them. Any sections of the tutorial designed specifically for group-interaction should be self-contained within a single component, so that individual users may omit those sections. In a group situation such as a tutorial class, a component-based approach gives the teacher freedom to spend as much or as little time as they would like on the WBT, and enables computer content to be interspersed with other media in order to maintain interest, maximise the benefits of socially-interactive learning, and reduce fatigue. Such an approach to design also clearly lends itself to components linked to other components (and even components forming parts of other tutorials), to facilitate the integration of related concepts, which is, after all, what the WWW is all about.

The 'Reasoning and Argument' WBT aimed to produce an interesting, interactive, self-directed, and widely accessible tutorial format for the presentation of important ideas. The tutorial was presented as a 90-minute activity with minimal direction from tutors, followed by a 30-minute illustration of concepts using video clips. Student feedback was collected via questionnaires, which indicated an overall positive experience of the tutorial on the part of students. Notably, students enjoyed the WWW as an educational medium, and did not report any difficulties in navigating through material, however some students did feel that 90 minutes was too long as a continuous computer-based session, and that they were presented with more material than could comfortably be absorbed in that time. A feature of student responses was the degree to which social interaction with other class members contributed to the overall tutorial experience. As a result of the feedback it became apparent that the way in which individual students most effectively use teaching resources such as these is quite different to the way in which individuals in groups may do so. Perhaps more than other educational media, the WWW is a medium in which the need to cater to both individual and group users is a vital concern. It was suggested here that the development of component-based content may solve a number of potential problems by providing flexibility in terms of what, how much, and in what order, information is received.

With the above considerations in mind, a slightly revised version of the 'Reasoning and Argument' tutorial is planned for next year's psychology course. The aim of revisions is to improve students' enjoyment and absorption of information by breaking the tutorial into components, and spreading the content over a two-week period. Given its strong social aspects, the early sections of the tutorial are planned for incorporation into a general introduction to the psychology course and to the departmental computers. This would have the benefits of introducing the students to the use of computers in the classroom and to the use of WWW-browsers for accessing information (skills becoming increasingly necessary throughout their university career), and would also provide an opportunity for students to meet other classmates while learning about concepts fundamental to psychology. This would free up some time for the second week's tutorial, which would incorporate the bulk of the present content. However, it is planned that the current content be restructured into discrete components, and rather than completing all of the material in a single session, computer activities would be interspersed with video clips, tutor-led discussion, and group work. It is hoped that this approach will address the problems of the current format by alleviating time pressures and fatigue based on long continuous computer sessions.

(c) Scott Gazzard and James Dalziel

The author(s) assign to ASCILITE and educational and non-profit institutions a non-exclusive licence to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. The author(s) also grant a non-exclusive licence to ASCILITE to publish this document in full on the World Wide Web and on CD-ROM and in printed form with the ASCILITE 97 conference papers, and for the documents to be published on mirrors on the World Wide Web. Any other usage is prohibited without the express permission of the authors.