CADAL Quiz: Computer Aided Dynamic Assessment and Learning Quiz

Angela Carbone*; Peter Schendzielorz
Monash University
angela.Carbone@cs.monash.edu.au

In 1996 the Department of Computer Science, Monash University, implemented a First Year Advanced Students' Project Scheme aimed at extending and stimulating its best first year students. The idea behind the scheme was to give students the opportunity to work on a project that best suited their needs and captured their interests.

One of the projects involved producing an educational package using the World Wide Web. A component which became known as CADAL Quiz, which is a multiple choice generator and assessment tool.

Unexpectedly, at the time, there were several academics wishing to move away from the traditional mode of educational assessment and move towards interactive, computerised tests.

As a result, the CADAL Quiz was incorporated into the First Year Computer Programming unit and utilised by lecturers, demonstrators and students.

This paper describes the work of an advanced first year student who developed CADAL Quiz. It highlights the unique quiz features, and its use by students and staff. The paper describes how the quiz was incorporated into the First Year Computer Programming unit and presents a conduit of attitudes useful to those who are planning to use the Web a resource for educational assessment.

ContentThe paper will be divided into the following major sections: 1. Introduction2. Design and description of the quiz3. How the quiz was incorporated into the first year Computer programming course4. Critical Issues - Statistics - Responses - Student perceptions - Tutor and Demonstrator perceptions - Results analysis5. Analysis and the educational impact6. Conclusion and Future directions

====================================================================

Extended Abstract1. Introduction----------------With the onset of the Internet, in particular the World Wide Web (WWW), it has been increasingly popular to move away from the traditional mode of educational assessment and move towards interactive, computerised tests [1, 2, 3]. Such a move was made easier by using the work of a devoted student who designed and implemented CADAL Quiz as part of the 1996 Computer Science, First Year Advanced Students' Project Scheme [4,5].

CADAL Quiz involved the design and implementation of interactive, dynamically generated, random quick quizzes on the WWW. CADAL quiz is now being utilised by first year lecturers, tutors, demonstrators and students to change the traditional mode of assessment to an interactive mode.

2. Design and description of the quiz--------------------------------------While there are tools to develop interactive lessons on the WWW [6,7,8] many lack the features of CADAL Quiz which makes it unique from others. These include:- 1.Questions can either be selected randomly or selection can be controlled2.Each question's A, B, C, D choices can be randomly ordered 3.Questions can contain text, graphics, animations and sounds 4.A particular quiz can only be answered once by the user 5.Detailed statistics, including time taken are compiled immediately for analysis 6.Results are optionally emailed to important people 7.Results are displayed to the user but can be turned off 8.Online password protected software to add questions and view statistics 9.Most information can be either specified at run-time or hard coded

CADAL Quiz was introduced into the First Year Computer Programming unit with modest goals in mind, such as:- 1. Cutting the cost of assessment, 2. Reducing the opportunity for students to copy and cheat in tests, 3. Allowing students to work on the questions at a time that is convenient to them, 4. Providing students with immediate feedback to answers, 5. Offering assistance and references to lecture material, 6. Maintaining student results for tutor and demonstrator feedback and curriculum development

3. Incorporating CADAL Quiz into the First Year Computer Programming unit---------------------------------------------------------------------- With traditional practices of assessment in the First Year Computer Programming [9], there has been concern about students copying preparation work and whether the mid semester test was cost-effective given the current pressure on resources and budget cuts. This year, CADAL Quiz changed the way in which students were assessed.

Twenty first year tutors and demonstrators devised and submitted five multiple choice questions each into the database. These questions were aimed at testing whether the students had adequately prepared for their laboratory exercise. By week 3 of the first semester students generated and attempted a unique CADAL produced quiz during the practical class and received the time taken to complete the test and a score out of 10. The results were automatically mailed to the demonstrator and counted towards 3 preparation marks. During week 5 the students were familiar with the operations of CADAL Quiz and as such the lecturer replaced the mid-semester 1, multi-choice test, by CADAL Quiz.

4. Critical Issues - Statistics - Responses -------------------------------------------- Perceptions from random groups of students, tutors, demonstrators and lecturers of CADAL quiz will be reported in the final draft. These responses will serve as a representative conduit for educational change from the traditional practise.

5. Analysis and the educational impact--------------------------------------This section will examine the impact that CADAL Quiz had on helping students prepare for their practical exercises. In particular how often did students use CADAL, and was CADAL useful in guiding them to prepare for the practical exercises.

Teaching staff were interviewed to gain feedback on any perceived benefits and pitfalls of the Quiz. The effect CADAL Quiz had on the operations of the laboratory classes and the feasibility of designing and shaping educational assessment tasks by a group of demonstrators and tutors will also be discussed.

Student results generated from the quiz determined CADAL's usefulness in steering course design. Detection of the hardest questions (ie. most wrong answers) and even which wrong answer is most commonly selected allowed staff to accurately locate the most misunderstood topics, right down to the wording of a possible answer.

6. Conclusion and Future directions--------------------------------------CADAL Quiz is an application of WWW technology that has had a significant impact on educational assessment materials on the Internet. The quizzes are interactive, dynamically generated and are available at the student's convenience.

The feedback from the quizzes provide students with personalised feedback. Tutors and demonstrators can analyse class statistics and lecturers can generate aggregate statistics automatically. These features may be used to tailor courses based on the student's learning patterns.

7. References-------------[1] Godfrey R, "The Word Wide web: A replacement, displacement, supplement or adjunct of traditional methods?", Proceedings of the Thirteenth annual Conference of the Australian Society for Computers in Learning in tertiary Education, pp 221-234, Adelaide, Australia, Dec 1996[2] Conway, D., Improving Educational Outcomes with Computer-Mediated Presentation, Proc. 1993 International Conference on Computers in Education (ICCE'93), Taiwan, pp.325-327, 1993 [3] Conway, D M, Student Responses to Hypermedia In Lecture Theatre: A case Study, Proceedings of the 1994 world Conference on Educational Hypermedia (ED-MEDIA'94), pp 141-146, Vancouver, Canada, July 1994[4] A Carbone, "Retaining the Best Students - An First Year Advanced Students' Project Scheme", Proceeding of the Second Annual Australasian Conference on Computer Science Education, (ACSE'97), 3-5 Jul, 1997[5] A Carbone, "The Evolution of a First Year Advanced Students' Project Scheme", (Tech 97/314) Dept. Computer Science, Monash University, Mar 1997[6] Sloane, A.M. and Dyreson, C., "An Interactive Self-Paced Learning Environment for the World Wide Web", Proceedings of the First Australasian Conference on Computer Science Education, pp.344-351, Sydney, Australia, July 1996[7] http://dialspace.dial.pipex.com/town/plaza/fg50/#list1 [8] http://www.best.indiana.edu/QuizSite.html [9] "CSC1011 Computer programming", Lecture Notes, Ann Nicholson and Graham Farr, Mar 1997