Full Paper

Lara Strassburger

l.strassburger@mailbox.uq.edu.au

Educational Multimedia Services, Teaching and Educational Development Institute

University of Queensland

In this paper I discuss the features of BrainZone, a new online assessment tool: the flexibility of its underlying structure and its reporting features are of great significance to the tertiary community. BrainZone is an excellent tool for delivery of formative assessment which makes it a valuable learning tool for students. It can also be used for summative assessment, and some of the advantages and disadvantages of this use are mentioned. Encouraging data, available from a small study, has shown that BrainZone use is likely to result in higher marks for the students who use it. However, it is essential to note that BrainZone will be of no benefit to anyone unless the questions and feedback that the lecturer or tutor create are valuable and well-constructed.

In the past, learning and assessment were quite distinct features of one's education. Assessment was viewed as a tool to verify that learning had taken place by the end of a set period of time. It was viewed as having little to do with supporting the learning process (McDowell, 1995, p. 302). It was believed that activities or exercises which were not assessed would not be completed by students. However, as our results of student use show, this is not the case. Students are keen to learn and they are more than happy to use innovative tools for their study. They are also very keen on the flexibility that control over their own learning strategies gives them, provided that they still receive support from their lecturers. BrainZone allows lecturers to provide this support while allowing student control over "when, where, how and at what pace they learn" (University of Queensland Working Party on Flexible Delivery in Teaching, 1997). One aspect of this flexibility is that students can control the place and time of assessment. Flexible delivery places the focus of education into a "learner-centred" mode in which a great deal of learner choice goes on and the teacher acts as a facilitator rather than a disseminator of information (UQ Working Party, 1997).

Some questions currently being tackled by tertiary institutions worldwide are "Should we be monitoring the progress of students in large classes and the progress of distant or external students? How do we do it, particularly when we're dealing with increasing numbers of students and decreasing numbers of staff and resources? How do we become true facilitators in this sort of environment?" BrainZone offers a way to deal with some of these issues and holds great potential for the future.

For each individual subject registered, lecturers or tutors can use the staff access area to:

• administer student details;
• create topics;
• create and edit questions (multiple choice, two types of short answer - keyword or specific, and drag-and-drop), answers, marks and detailed feedback;
• include high resolution colour graphics in any question;
• create, edit and trial assessment exercises; and
• monitor students' progress with each exercise.

BrainZone offers numerous options not found in similar packages:

All tests are composed of questions which are stored under topics within one subject. This provides a range of organisational options (Figure 1 highlights the basic organisational structure provided in BrainZone - within it rests the flexibility offered by the package).

There is a high level of flexibility in:

• level of feedback for individual questions and for entire assessment exercises (eg - provision of whole marks, partial marks and/or detailed textual feedback);
• style of assessment exercise - formative or summative, randomised or set-question from one or multiple topics (eg - A formative, or 'learning mode', test made up of randomised questions from one single topic called 'Week 1 - Cell Structures');
• availability of assessment exercise - to/from dates and time limits (eg - A test for summative assessment can be made available during tutorial times only);
• options for marking answers (eg - partial, full or no marks for any particular answer); and
• student/test reports - test reports, student reports, test-attempt reports (eg - To get an overview of how well students are doing on a particular test you get a test report; to then see how one individual student has done on all of their submitted tests you get a student report; and to see how that student has answered individual questions on a particular attempt at the test, then you get an attempt report).

The reporting systems described above allow lecturers and tutors to monitor students' progress through the assessment exercises. It also provides insight into:

• the study habits of students;
• the questions with which students are having the most success; and
• the questions with which students are having the most difficulty.

The reporting systems also enable the immediate revamping of assessment questions as necessary. There is a direct link to the question editor, which opens in a new window containing the specific question to be edited.

Students who are given the opportunity to perform assessment exercises in BrainZone are offered the benefits of:

• self-pacing;
• mastery learning by working through formative exercises many times (a number of students have been recorded working through an individual exercise over thirty times)
• progressive self-monitoring;
• instantaneous feedback mechanisms; and
• working in a non-threatening environment.

It is a package that is engaging and fun to use and one which encourages students to be excited about the subjects they are studying.

The wide variety of features offered in this first version of BrainZone, combined with well thought out content, enable it to be a highly effective tool for the delivery of student-centred learning materials. Student-centred learning is the heart of true flexible delivery: it "empowers students to take increased responsibility for their own learning" (Tertiary Education Institute, 1996, p. 91) which therefore makes education "not the passive receipt of knowledge but an active and radical owning of knowledge by students" (Tertiary Education Institute, 1996, p. 91). This empowerment is the key underlying potential offered when BrainZone is used effectively, making it, and its future developments, an essential educational tool at the tertiary level.

Studies show that formative assessment, or "assessment for learning" (Boud, 1990, p. 101) designed to meet the needs of students (rather than for marks) promotes more meaningful learning and improves learning quality. In combination with appropriately designed questions in BrainZone, students should be given "encouragement, response and feedback on what they do, as appropriate, with a view to them becoming more effective in their learning" (Boud, 1990, p. 101). Well designed feedback, not just 'Correct!' or 'Sorry, wrong answer', is essential to make BrainZone an effective and meaningful student learning tool.

The use of BrainZone allows students to get into good study habits, encouraging them to study over the course of a semester and get out of the 'cramming' habit. This promotes the use of long-term memory and allows time for comprehension and interpretation as opposed to the use of short-term memory solely for the purpose of an examination mark. Because of its WWW accessibility, students have the opportunity to access the assessment activities an unlimited number of times at any time of the day. Its innovative approach and exciting interface encourages use and allows students to perform the activities at their own pace and in their own space.

A large percentage of students are still entering tertiary institutions with a fear of technology and a dislike of computers. Increased use and exposure to computerised tools, like BrainZone, is the best way to dissipate these fears. Students will find BrainZone easy and fun to use and students who are given the opportunity to use BrainZone in combination with well-designed content will find that they have gained something from it and realise that other technologies will benefit them as well. Studies done with a similar package have, in fact, shown that apprehensive computer users come out "greatly reassured" and show increased confidence in computer use upon completion of the examinations (Pritchett and Zakrzewski, 1996, p. 246). The package on which the studies were based was designed for marked examinations only, though the students were given ample time prior to the exam day to work through practise tests.

As previously mentioned, BrainZone can be used in both learning and examinable modes. One of the ways in which it can be used most effectively is to provide a series of short randomised tests based on weekly topics in 'learning mode', and then to follow this with a much longer test in 'examination mode' which uses randomised questions from all of the weekly topics.

Unfortunately the difficulty in using this method is security; although we have access codes and passwords in place, as well as date and time limits, it is difficult to know for sure that a student has really completed his or her own examination. It would be easy for a student to give their access code and password to another student who could take the test for them. However, ways around this are to use tutorial/discussion group times to run examination sessions in monitored computer labs. This, as you would expect, would depend on the facilities available at the site in which BrainZone is being used, but it is an option worth considering.

BrainZone, in the long run, can ease the time commitments required of lecturers and tutors, allowing them more time for student contact, curriculum development and implementation of formative assessment activities. For use as a formative assessment tool, it is important to realise that the creation of high quality questions and useful feedback is not a quick process, but that once the questions have been created they can be reused and repurposed as needed. Additionally, when BrainZone is used for marked examinations, lecturers and tutors will be freed of the time commitments involved in marking that are usually required of them. This can be a major advantage, particularly in large classes.

Currently under trial in over 70 subjects, an extensive evaluation of BrainZone has not yet been completed. However, Dr Alison Bailey and Bev Oelrichs, Department of Physiology and Pharmacology, University of Queensland, were involved in a BrainZone trial during the final two to three weeks of first semester, 1997, and have provided the data found throughout this section.

BrainZone was introduced into one first year subject which was taught jointly by the Department of Anatomical Sciences and the Department of Physiology and Pharmacology, and into second year Physiology subjects which had lectures in common. The first year subject had approximately 1300 enrolled students while there was a total enrolment of nearly 500 in the second year subjects. The tests consisted of randomised multiple choice questions; feedback to students was provided both as a score and as comments explaining the correct responses.

A total of 416 first year students responded to a questionnaire about their use of computers and departmental computer facilities. Of these, at least 91 had used BrainZone (though 177 students left this question blank altogether) and rated it on a 1-5 scale as shown in Table 1.

As anticipated, the best students used the tests the most (as shown in fig 2) which largely reflects the higher motivation levels and better study patterns of students who chose to use the tests, but, as you will see, the results are encouraging enough to promote BrainZone use to all students.

When students' end of semester mark in the first year subject was related to their Overall Position (OP, score for tertiary entrance with a high mark of 1) (fig 3) and the end of semester mark in the second year subjects was related to the end of semester mark in first year (fig 4), it was clear that the best students performed the best on the exam. This was expected. However it also revealed that all students who used BrainZone did better than their counterparts who did not, including the students with an Overall Position of >10.

Even when the motivational factor is excluded, there is evidence that the tests significantly improved final examination marks. BrainZone tests were written first for the topics typically considered most difficult by students. Consequently it is not surprising that students who did not make use of BrainZone did significantly worse on the exam in the topics covered in BrainZone tests. They also did not perform as well on the topics not covered in BrainZone, whereas the BrainZone users did quite well.

If BrainZone tests in only a limited number of topics which are introduced very near the end of a semester can make this significant a difference, we expect BrainZone tests to be very valuable for students when all topics are covered and are available early in the semester.

While a few students used only one test once, the average was from 5-15 attempts with final attempts being completed in 2-3 minutes. A few students made considerably more attempts than the average. One first year student did the two tests a total of 200 times. While this may seem excessive, this student achieved the top grade on the final examination despite the fact that her OP entry score was at level 3.

These first results are very promising and as we look to the future of BrainZone we are looking to incorporate a number of innovative testing facilities not currently available. BrainZone will soon be suited to an increased number of disciplines. Some of the things we hope to do with this product include:

• incorporation of the drag-and-drop question type into the question editor;
• ability to incorporate video and sound into the questions; and
• development of peer assessment facilities in which student groups can be set up for essay evaluations. There can be an objectives checklist (set by students or lecturer) and commenting functions enabling specific comments to be inserted at points throughout the essay.

With well-constructed content, the use of BrainZone by students will help to change study habits and increase learning capacities. It will help students enjoy studying and will help to decrease their fears of computers and technology. It will help lecturers to keep track of the progress of individuals in large or external classes throughout a semester and will help lecturers to manage time more effectively with long-term use. As the package is developed further, we expect its value as a tool in all disciplines to increase significantly, with the expectation that it will become a widely used tool in universities Australia wide.

(c) Lara Strassburger

The author(s) assign to ASCILITE and educational and non-profit institutions a non-exclusive licence to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. The author(s) also grant a non-exclusive licence to ASCILITE to publish this document in full on the World Wide Web and on CD-ROM and in printed form with the ASCILITE 97 conference papers, and for the documents to be published on mirrors on the World Wide Web. Any other usage is prohibited without the express permission of the authors.