The Computer Aided Testing System (CATS) is a software package designed for interactive and classroom monitoring of student learning at any level for any subject. Tests are efficiently generated and marked. Diagnostics include the production of concept maps, learning curves and National Profiles reports. It is a powerful classroom management tool for teachers.
The Computer Aided Testing System (CATS) is a software package designed for interactive and classroom monitoring of student learning at any level and for any subject. This classroom management tool offers:
In today's schools there is a great diversity of teacher methodologies and availability of resources. In some situations there are competent teachers using sophisticated information systems with highly motivated students while there are other teachers struggling with poor student behaviour and inadequate resources.
Teachers are expected to know the current level of achievement of each student in comparison to the course objectives, what may be hindrances to learning and if the students are actually progressing.
Students on the other hand are more highly motivated when they know in detail how they measure against the objectives, are given rapid feedback regarding attempts at learning, are given opportunity to improve, and are assured that they are actually progressing.
With the recent introduction of the National Profiles in Australia, teachers have the opportunity to comprehensively map the progress of students. It would be of considerable advantage to teachers to be able to have some assistance in making judgements about students' levels of achievement and their progression through the profiles. It would be even more advantageous if this could be automatically achieved during the normal classroom assessment process and results be made available immediately on demand.
To achieve this, much assessment must occur, be recorded and analysed. Teacher time involved may be extensive.
The analysis of assessment items and the subsequent transformation of this into effective diagnostic decision making are most often the first tasks to be abandoned or shortened under the pressure of a normal classroom teaching load.
The CATS program has been developed to meet these needs.
Question analysis indicates validity of the questions.
This information allows teachers to rapidly diagnose strengths and weaknesses in the class (or individuals) and make effective judgements for revision, reteaching or course evaluation.
The learning curve is the best fit line (least squares regression) of the average concept achievement over time. It is generated by considering the aggregate of successful encounters with each concept over the period of recorded testing. It is not merely an analysis of test scores.
The learning factor is defined as the slope of the learning curve. A factor of 0 indicates that no learning has taken place, +1 or above indicates maximum learning.
A learning curve indicating good progress.
Concept map of a test on legumes.
Experience has shown that students enjoy doing the tests, particularly those with questions based on diagrams and pictures. Rapid feedback is important. During lesson time students use the computers in groups of two or three and work through the formative interactive tests set for them. Much discussion ensues as they debate the relative merits of the choices presented. Answers are not given but on completion of the test they are presented with a range of diagnostics which assists them to deduce which questions are giving difficulties. They may repeat the test and are only allowed to proceed with the next test when they obtain a perfect score for the current test. There is great enthusiasm when the learning curve shows improvement.
As an assessment test approaches, the students are aware of the results they may expect and know the areas they need to concentrate on to achieve best results. The teacher is in demand to explain specific weaknesses.
Weaker students are enthusiastic to progress because they know that the teacher is concerned about an improvement in their learning curve rather than the actual score in tests. More able students are motivated to keep the learning curve increasing.
For example, the pre-test consists of 30 questions and covers most identified concepts in the objectives, It has knowledge, comprehension and higher order questions with a range of difficulty and discrimination factors. At least some questions are included which are linked to National Profiles. It takes about 20 minutes to prepare and print out ready for photocopying. The summary is perused and adjustments made to provide a balanced test.
The test is given in the first lesson of the new area of learning and students mark their answers on the prepared answer sheet. The test is completed quickly, the teacher sets a small task for the class, types the responses into the computer and advises students of the results. Individual result sheets are printed out before the next lesson for each student showing marks, percent, responses and a concept map showing the strengths. Students who are absent sit the test interactively in the next lesson and results are immediately available.
Diagnostic tests are sometimes administered in the same way and sometimes taken interactively. When students are progressing at different rates or on different topics then interactive tests are set to meet their needs and are taken when ready. At the end of the day the teacher scans the student files to gain a knowledge of attempts students have made and how they have achieved. Some diagnostic tests have been used to evaluate the effectiveness of learning tasks set for homework. A class report produces a concept map for the whole class showing the area of maximum difficulty . With this, the teacher plans a short targeted re-teaching of the concepts least understood by most students in the class.
In this class, the measurement of learning is an important factor of the assessment process. Students of all abilities who see specific evidence that they are progressing well are encouraged to apply themselves more diligently to their studies.
At the end of the learning unit, an analysis of the National Profiles shows that students have made progress in their total learning against this standard although no specific testing has been carried out to achieve this.
ACER Newsletter. (1993). Towards National Agreement on Learning Targets. (76) .
Neill, N. T. (1993). Computer based Testing with Question Mark Professional. Computer Education, (74) 23-26.
Ring, J. (1993). Characteristics of state of the art computer administered testing systems. Australian Educational Computing, 8(2) 25-28.
|Author: Barry Peek, Department of Education and Children's Services, South Australia
Please cite as: Peek, B. (1994). Measuring learning and increasing productivity with the Computer Aided Testing System. In J. Steele and J. G. Hedberg (eds), Learning Environment Technology: Selected papers from LETA 94, 234-237. Canberra: AJET Publications. http://www.aset.org.au/confs/edtech94/mp/peek.html