Academic Development Unit
J.Hellwege@latrobe.edu.au |
| Academic Development Unit
|
The Web is being hailed by some as a transforming power for education in the leadup to the 21st century. Clearly we can now diversify and improve the ways in which course material is made available to our students. We can design and implement systems of education to deliver this knowledge to large numbers of students. However, is this an advance in educational terms? What are studentsí attitudes to course material available via the Web? More importantly, are they learning the material more effectively? This paper will report on the evaluation of a first year Geology subject at La Trobe University.
1 Shifting Paper to PowerPoint
First year Geology is delivered to over 100 students at the central Bundoora campus and 30 students at the remote Wodonga campus of La Trobe University. Lectures are available to studentsólive at Bundoora and via videotape at Wodonga. In 1995 a project was commenced to covert the entire body of lecture resources (lecture notes, slides and diagrams) into computer slideshows, using Microsoft PowerPoint. Once all the disparate lecture materials had been organised and centralised into PowerPoint slideshows, they were then projected during first year lectures at the Bundoora campusóusing a computer dataprojector connected to a laptop computer. Paper lecture notes are no longer handed out during lectures.
Although the lecture staff involved
with the creation of the slideshows disliked the investment of
time and effort which was required for their production (a ratio
of about 12 hours of production time for each hour of lecture
time), they were quickly impressed by the benefits which the new
system brought to their teaching.
Teaching staff who have shifted to
the new system have reported that they find their students to
be dramatically more attentive during lectures. It is unclear
as to what specifically has altered the behaviour patterns of
the students; it could be the more organised and focused point
format of the computer slides, it could be the impact of colour
projection, it could also simply be the gloss of ëHigh Techí
and aura of ëCorporate Professionalismí which the
lectures have now acquired.
Interviews and feedback from the students themselves has shown that they feel that the concise and focused point by point format of the slideshows provides an easily absorbed skeleton for their understanding of the concepts being presented. They no longer are obsessed with the lemming-like urge to inscribe reams of paper with cryptic lecture notes in an effort to capture the ebb and flow of the lecturerís thought patterns. The students are more relaxed and focus their attention upon understanding the topics of the lecture rather than practicing their dictation skills.
The lecture slideshows incorporate a large number of high quality, computer-generated, full colour illustrations of each key concept. In some cases, computer animations and video clips have been employed as well. This wealth of images has been referred to by the students as a major aid to their understand, and significant improvement over the hazy monochrome graphics which they are used to in the photocopied overheads of other courses.
Another key benefit of the new system has been that the lecture materials are no longer chained to an individual presenter. The slideshows (far more organised than the previous cobbled collections of overheads, slides and scribbled notes) can easily be presented by members of staff other than the materialís author. Each new computer slide prompts the presenter on the next concept to be covered, and the order in which to discuss each topic. This new portability of the lecture materials has meant that should the designated lecturer be unavailable for a particular lecture, a substitute can be organised with comparative ease, and the lecture series proceed on schedule. The issue of portability is vital where distance teaching is involved. The computer slideshows have meant that students on remote campuses can attend lectures identical to their metropolitan counterparts, even if two different members of staff are presenting the lectures.
2 Shifting to the Web
The key goal of the whole project was
to direct the students attention away from pieces of paper sitting
on their desks and onto the concepts which the lecturer was wishing
to convey. One part of fulfilling this aim was the decision that
paper lecture notes were to no longer be handed out during the
course. The obvious trap with this move was that the students
would simply turn the lectures into an exercise in transcription.
Clearly we needed to give the students the confidence that it
was safe to concentrate upon the ideas of the lecture, rather
than struggle to capture the entire information content of the
lecture onto paper.
To instil this confidence in the students, it was decided early in 1996 that the next step in improving the dissemination of the lecture materials was the creation of a World Wide Web site. The aim of the website [http://www.adu.latrobe.edu.au/Geology/] was to take the PowerPoint computer slideshows and make them available for the students to peruse by computer from anywhere at anytime.
The Geology Website makes available the entire body of the PowerPoint slides for immediate viewing. Each of the lectures has its own Web page on which are placed thumbnail images of the PowerPoint slides for that lecture. Students can simply click upon the thumbnail of the slide they are interested in, and they are presented with a GIF image of the desired slide.
3 Student feedback
One of the first features fitted to
the website was a form which enabled the students to anonymously
send feedback on the website and what they thought of the whole
shift away from traditional teaching methods. On the whole, opinions
ranged from the mildly favourable to the wildly enthusiastic;
there was very little negative response to either the website
or the project. Overall they feel they have markedly increased
their understanding and retention of the subjects being discussed,
thanks to the shift in teaching method and the improved availability
of the materials. Typical responses include:
This is certainly the way to go.
It gives the student time to ingest the information at a pace
conducive to retaining the info.
Accessing the online geology lectures
is great. This means that in the actual lectures we are able to
learn, rather than writing and not understanding. I find accessing
the notes Iíve understood more of the subject. Keep it
on..!!!!!
The few cases of negative feedback which we have received can mostly be grouped under the heading of ëTechnological Teething Troublesí. Although most of the students seemed to have no difficulty in navigating the World Wide Web, there were some complaints about issues relating to download times and printing speed. Upon examination it was found that the website was frequently being accessed from a campus computer lab which was equipped with computers woefully inadequate for accessing the Geology site (or any other site on the Web for that matter). Consequently, advice was given out in lectures as to where better quality computer labs could be found, and a hardcopy version of the slideshows was made available in the University library for those who were unable to break their addiction to photocopier fumes.
Student feedback has revealed that the ability to look through a lectureís content in advance is greatly appreciated, and further enables the students to focus upon the concepts rather than the content when they attend the actual lectures. The more advanced and enthusiastic students are able surf through future lectures and get a feel for where the ideas they are currently studying lead, where they fit into the overall picture of the subject. Students can also explore which topics they would like to do some preparatory spadework on before the inevitable assignments arrive.
4 Enriching the Web resource
In order that the students can review the slides when they donít have access to the Internet, the slides for each lecture were made available for downloading in Adobe Acrobat format (PDF files). The Acrobat versions of the slides are intended as the real replacements for paper lecture notes. They can be downloaded, kept on disk, viewed and searched from any computer (Mac or PC, as the PDF format is completely cross-platform), anywhere, anytime. Students initially were laboriously printing out the PowerPoint slides for each lecture; however, a close examination of the Web siteís logs reveals that the students quickly progressed to downloading and using the Acrobat (PDF) files, rather than relying upon paper copies.
To ensure that the intellectual property of the lecturers is protected (and that the education which the students have paid for is not openly available to anyone with a computer and a modem), the lecture materials can only be accessed once the person visiting the site provides a valid username and password.
In designing the Web site, a guiding principle was that the online material had to be superior to paper notes. It is our belief that putting straight text on the Web would not have been motivational and would have simply been another hurdle placed between the students and a pile of paper lecture notes. The colour illustrations, animations and electronically searchable PDF files are a significant part of that superiority to paper notes. The other half of the equation is exploiting the wealth or resources already available on the Web. Links to other geological websites, downloadable tutorial software, and an online dictionary of geological terms are progressively being added to the website. These ëvalue-addedí resources encourage the students to make use of the website and explore the subject beyond the borders of what can be expressed within a one hour lecture.
Perhaps the most important aspect of the website is that it give the students extensive experience in developing the skills needed to access and search and manipulate electronic / online materials. These skills are becoming vital for students from both a research and employment standpoint. The time is almost upon us when academic departments would be failing in their responsibilities to their students if they did not provide them with these sorts of research skills.
5 Key strengths of the system
Overall, the key points which have motivated
the inception and development of the slideshow / website project
have been:
6 Delving into the logs: What strategies have students developed to utilise this online resource?
Whenever someone accesses one of the Geology web pages, a record of the time, the address of the computer requesting the information, what pages were viewed and what files were downloaded, is written to the log file of the Web server. The log files covering the complete period of time since the websiteís launch contain some 150,000 lines of entries, and research on their contents by lecturers and the site administrator can yield up valuable information about how students are accessing the website.
The logs can provide answers to such
questions as:
All of this information provides valuable insights into how the students utilise the site and how it should be modified for the next intake of students.
Even a cursory glance at the log files reveals that a surprising number of students are connecting to the website from home via modem. As the lectures are locked behind password access, any connections to the site from computer addresses ending in the ë.COM.AUí of commercial Internet Service Providers, or after-hours connections made via particular on-campus computers which route traffic from the Universityís modem pool, can only come from students pursuing their course from home. The frequency with which these connections appear in the log files indicate that the time for this type of study resource has finally come; that the ëcritical massí for students with computers connected to the Internet has been reached.
The ability to connect to the site from home has meant that the students are accessing the material at a wide variety of times. Although teaching staff may question the wisdom of studying the website at 2 am, students have indicated that the ability to grab a key piece of information when assignments are being completed is a valued safety-net. Interviews and feedback has shown that this flexibility in ëworking timeí is greatly appreciated by the students.
Despite the activity in accessing the website from home, the main body of connections are being made during business hours from computer labs throughout the campus. While this fact is not particularly remarkable in itself, it does have important guiding implications for the provision of computing infrastructure within the University. Clearly the creation of study resources such as this cannot proceed on a wide scale unless there are adequate computer lab facilities within not only the departments involved, but across the University as a whole. Although the Geology department has made considerable investment in providing computers for its students, it appears from the logs that the students prefer to access the website from machines where they are not so readily under their lecturersí watchful eye. In the first weeks after the launch of the website it became clear from the logs that the students were most frequently using labs on campus which were highly visible, but which contained the slowest computers. Following some guidance given in the lectures, it could be seen that students began to disperse to other locations on campus which offered them superior machines and better access to the website.
A more detailed examination of the log files starts to reveal patterns (or sometimes, a lack of them) in the way in which the students have been interacting with the website. The first thing which has been noticed is that the students are not accessing the Web materials according to a strict regimen, but are using them to refresh and bolster their understanding of key lectures (with complex subject matter). There are relatively few examples of students methodically starting at ëLecture-1, Slide-1í and proceeding through to ëLecture-20, Slide-20í. Instead, a typical session will involve accessing the site and picking out a few key slides from the most current lecture, next the students will often return to a selection of slides from previous lecturesóobviously seeking to reinforce key concepts of the more difficult topics.
Further to this, there is no firm pattern to the way in which lectures are accessed on a week to week basis. Particularly difficult lectures are accessed heavily while more pedestrian topics are left relatively untouched. For example, a lecture covering the origins of continental crusts has been visited more than twice the number of times that a lecture on ëOrogenesis and Plate Tectonicsí has, despite both lectures being only a few days apart, and covering similar conceptual ground. Clearly from this great disparity in the number of ëhitsí between the two lectures, it can be seen that the website is being used as a tool to target and bolster understanding of specific areas of knowledge (a hypothesis subsequently backed up by interviews with students), rather than as means of blanket revision or as a substitute for attending the real lectures.
The statistics for the number of hits to the website on a day by day basis reveal that although there is a gradual and steady increase in usage of the Web materials (as the students progressively master the technical aspects of navigating the Web, or discover the value of the lecture resource) as the months progress, accesses to the website surge dramaticallyóincreasing by some 80% to 100% above the preceding weeksóduring exam periods and when major assignments are being prepared. From the statistics it can be seen that the students are employing the Web materials as a fundamental reference source and a means of ërelivingí critical lectures during intense study periods. In this situation the website performs one of its most important functions; solving the age old student dilemma of covering the knowledge from lectures missed through illness or due to other reasons.
The log files can be examined for elements other than the simple ëWhat pages were visited?í. Of particular interest has been analysing the logs to find records of files being downloaded, and hence determining the patterns of usage for the Acrobat PDF files. Log information and student feedback so far have indicated that the students place considerable value in the PDF files, as they provide them with a means of building their own selected ëlibrariesí of lectures for referencing from home without the technical complexities (and time delays) of accessing the website via modem. Anecdotally, the popularity of the PDF files with the students was most keenly felt when, during a hiccough in the production of the site, a flurry of emails poured in via the siteís feedback form asking what the delay in the supply of PDF files was and demanding that they be made available immediately.
The story which the log files reveal is very similar to that given for how students are accessing the lecture slide GIFs. There is a very close correlation between which PDF files the students choose to download and which lecture pages they focus most of their attention upon. Obviously the PDF files are being employed in a similar way as the website lecture slides; as a tool to boost comprehension of key topics. Student interviews reaffirm this viewpoint of how the PDF files are being utilised.
It is also important to notice in the PDF file log entries the rate at which the students adopted their usage. Download statistics for the early stages of the siteís development show that there was at first a considerable reticence in making use of the PDF files. Discussions with the students and early reports of students resolutely determined to print everything within clicking distance have indicated that the slow adoption of the PDF facility by the students was a combination of technical difficulties in installing the Acrobat Viewer software on their home computers and, more significantly, a cultural belief that paper is the only format that knowledge can come in. With the benefit of hindsight, it can now be seen that it would have been beneficial to organise a small workshop for the students which demonstrated installing and operating the Acrobat software, and proved to them that the PDF files could (when used correctly) be an equal and often superior medium for learning than the uninspiring non-colour, non-searchable, photocopied handouts which they are more familiar with.
7 How students at a distance campus use the online resource
The computer slideshows and Web facility are proving a boon to Geology students on the remote Wodonga campus, who for the first time have access to resources identical to those of their Bundoora counterparts. In previous years this was a difficult subject for staff 300 km away from their students to teach effectively, and the resultant student performance was disappointing. As will be discussed later in this paper, there is ample anecdotal and statistical evidence to prove that the new Web resources are a fundamentally excellent tool for supporting students on remote campuses. A fine-tooled examination of the logs and student interviews have revealed that Wodonga students have adopted similar patterns of accessing the Web site and the PDF files as their Bundoora counterparts.
The main areas of behavioural difference between the two groups is that the remote Wodonga students make far more limited use of modem connections to the website; due largely to the severe limitation in the supply of reasonably priced Net access in regional areas and the more restricted dial-in modem access available at the remote campus. The other notable difference between the two student bodies is that the Wodonga students have reported a tendency to make accessing the Web resources a group, rather than an individual, exercise. This makes better use of the limited computer facilities available and leads to group discussion of the topics presented.
8 Are students learning basic Geology material effectively?
The greatest vindication for the adoption of computer based lecture presentation and for making those same materials available via the World Wide Web is to be found in the pass rates for the courses at the Bundoora and Wodonga campuses.
During the period before the introduction
of the PowerPoint lectures or the website, the pass rates at both
campuses were of serious concern to the Department (table 1).
1993 1994
Bundoora 45% 53%
Wodonga 32% 41%
The pass rates for the remote campus of Wodonga was of particular concern as the Wodonga students did not have the support of frequent access to staff, and instead had to undertake the course via videotapes of the Bundoora lectures.
In 1995 the PowerPoint lecture system
was introduced, seeing a dramatic improvement in the Bundoora
passrateóa rise of 26% above the 1993 figure! In 1996,
with the introduction of the Web materials, the Wodonga students
finally had equal access to the same resources as their Bundoora
counterparts. The Wodonga passrate has improved to the point where
students on the remote campusówithout direct access
to teaching staffóare now performing marginally better
than their metropolitan counterparts. This single fact is
the most powerful argument for the adoption of Web-based teaching
resources. From a low point in 1993 of 32%, the Wodonga passrate
has now more than doubled to 74% (table 2).
1993 1994 1995 1996
Bundoora 45% 53% 71% 73%
Wodonga 32% 41% 52% 74%
Note that there has been no significant changes in either the course content, or the examination papers during the period in question. Thus, we can make these direct comparisons in the pass rates from one year to the next.
9 Where to from here?
Now that the PowerPoint and Web resources have been fully integrated into the courses at both campuses, and the projectís merit proven by the dramatic leaps in pass rates, the question arises as to where the project should go on from here?
One unforseen product of the project has been that the students who have had the support of the new teaching system during their First Year of University, go on into their Second Year courses, and are suddenly confronted by traditional teaching methods. The result has been that the students concerned tend towards having the typical ëFirst Year Experienceíódifficulties with adapting to independent study, high dropout rates, etc.óduring their Second Year of University. The evidence for this is thus far only anecdotal, but is of concern to the Department and is being monitored closely. The success of this project has encouraged the Geology staff to consider what appropriate use of these technologies can be made to enhance the whole degree program.
Regardless of how the project flows on to other courses, there are considerable improvements which will be made to the materials and their presentation, in light of the student feedback and the evaluation procedure.
Particular attention will be placed
upon introducing the students to the website, and ensuring that
they are comfortable with its resources, software, and the independent
and explorative study environment which it hopes to foster.
Note: Free access to certain demonstration
lectures has been provided on the website. From this we have received
valuable formative evaluation feedback from several institutions.
If you would like to sample the La Trobe Geology website, point
your webbrowser at:
http://www.adu.latrobe.edu.au/Geology/