IIMS 94 contents
[ IIMS 94 contents ]

Development of multimedia test system

Fujihara Yasuhiro, Shibata Shimako, Yoshikawa Yukari and Nagaoka Keizo
Kobe University, Japan
In the system we developed, both of output and input of computer are dealt with multimedia. The test is displayed with not only characters, but video and sound. The answer of testees is varied with writing the characters, selecting the alternatives, pointing in a video image, drawing, etc. Three examples are shown: 1) multiple choice item with video alternatives, 2) item of pointing time and place, 3) item of answering with voice. It is useful to use such a multimedia test to estimate the ability of the new achievement view like knowledge application and problem solving and so on.

Introduction

In this paper, we propose a computerised test system using multimedia technology. Two examples are shown in Figure 1.

Figure 1

Figure 1: Examples of multimedia tests

Background of test method

The test plays an important role in educational evaluation. In Europe, the test was put into practice at the age of Ancient Greeks (Chase & Ludlow, 1966). There was a practical behavioural examination for one's expression ability in Sparta, and an oral examination which was famous by Socrates in Athens. But, it was after 19th century that the paper and pencil test was frequently put to use in university. In 19th century, the paper and pencil test became in common with the advance of technology such as development of printing and price decline of paper and writing material. This enabled us to estimate one's ability more objectively compared with the last oral examination. Nowadays, almost of the test is practised with paper and pencil.

In the latter half of the 20th century, computer has come in and to be used as a test device (Linn, 1989). Some large scaled Item Bank Systems were established recently like SAT in USA Using computer in test enables to measure changing record of answer and response time of testees, which is very effective information at the feedback (Nagaoka, 1992). They have been not able to be measured in the pre-computer test era.

The latest trend of computer technology is multimedia. So far, the character was mostly used to make items in former computer test system, but now the utilisation of multimedia has become very easy to make items which are constructed with picture, sound and video with progress of the multimedia technology.

Appearance of new achievement view

In the traditional test, amount of knowledge was thought most important in educational evaluation. We will propose a better surrounding to put into practice such multimedia test as a method to estimate the abilities of knowledge application, problem solving and integrative understanding. In Japan, all school has a curriculum which is conforming to the Course of Study that the Ministry of Education announces publicly. The Ministry of Education (1989a) announced officially the new achievement view in the Course of Study. "The memory work was thought much of, but now the ability of willing to think, judge, express and create something is attached great importance."

We are developing a multimedia test system as one of the best way to estimate such new achievement view as in the Course of Study.

Advantages of multimedia test

The advantages of using multimedia for testing are as follows:
  1. Teachers can measure the learner's achievement of the educational objectives more correctly which were dealt with in a traditional computer test.

  2. Teachers can measure the learner's achievement of the educational objectives which had been difficult to measure in a traditional computer test.

  3. The change of test media from paper and pencil to multimedia will make a revolution of the educational objectives in the educational evaluation. New objectives will be raised which can be defined only with multimedia.

Utilisation of multimedia in test interface

In computer test, output information of a test item can be divided. for three parts: 1) question part, 2) condition part and 3) alternatives part. In opposite, input information is not divisible, that is only testees' answer at one time.

The device and information for output are shown in Table 1. In case of the paper and pencil test, the most part of output information is expressed with character. Such situation is the same in traditional computer test too. But it is easy to use picture, sound and video in multimedia computer test.

Input device Input information
keyboard character
pointing device such as mouse selection of alternative
tablet illustration
microphone voice
video camera action

Table 1: Input device and information

The device and information for input are shown in Table 2. In case of the paper and pencil test, we use pencil to answer, and the most answer must be expressed, by characters or marking. Up to now, in taking a traditional computer test, testees use in the almost case a keyboard or a pointing device such a mouse to input their answer. But in case of multimedia test, many kinds of input device can be conceived.

Output device Output information
display character
display picture
loudspeaker or headphones sound
display, loudspeaker or headphones video

Table 2: Output device and information

There was a part of multimedia test, as answering by drawing, or setting a problem with voice. We propose such surroundings as can be done such test that has been dealing as special. In short, testees can take all types of test as in chapter 3, by using computer, we can manage such multimedia data in a lump. We am to built multimedia data base by extending the way of thing of Item Bank, and are to deal with multimedia data.

Classification of the type of multimedia test item

Figure 2 shows a classification of multimedia test items by the way of testees' answer. We firstly classify multimedia test items for two as Selection type and Expression type. In the former type, the way of learner's answer is a selection of alternatives or a pin-pointing on a picture by pointing device such as a mouse. In the latter type, using other kinds of devices shown in Table 1. The former has a feature that objective marking is possible, namely, automatic marking by computer. Contrarily, the latter is difficult to be marked by computer. In the former type, the testees' answers are recorded as numeral value (the number of alternative, the coordinate in picture and so on) and they are able to be estimated by function. In the latter type, interpretation of testees' answers are not able to be dealt by computer. This classification corresponds to the traditional classifications, Objective type items and Essay type items. Recall type items represented by Short Answer form usually include Objective type, but Recall type is classified as Expression type in multimedia test according to above mentioned standard.

Figure 2

Figure 2: Classification of multimedia test items by way of testees' answer

Selection type is classified Recognition type and another type. The former is the type of which the alternatives are shown (for example, Multiple Choice form, True-False form, and so on). The latter is the type of which the alternatives are not shown. We term the latter Pointing type items and it is characteristic in multimedia test.

Example of multimedia test in Life Environmental Studies

Characteristic of Life Environmental Studies

The latest Course of Study was revised in 1989. One of the highlights of this revision is that Life Environmental Studies took the place of Science and Social Studies in the first and second grade of the elementary school. Life Environmental Studies aims "To educate those who have social concerns and natural concerns through the practical social activity or experience, and in the course of that, learn the custom or skills which is essential for one's life and make it one's own and lay the foundation enough to be able to stand on one's own legs." (Ministry of Education, 1989b).

The difference of Life Environmental Studies from traditional subjects is as follows:

  1. It attaches great importance to cultivate self expressive ability for pupils and to notice the relation between themselves and nature or society.

  2. The lesson is often practiced out of a classroom, and experiences are took seriously.

  3. The textbook has also a lot of beautiful pictures and has a few words, which means that the Life Environmental Studies does not take a serious view of character very much.

  4. It is very difficult to evaluate by traditional test method.
In Life Environmental Studies, educational objectives are quite different from those of the former subjects. So now the new view and way of evaluation is seriously needed. Above all, we think that Life Environmental Studies is one of the most suitable subject to be applied multimedia test.

Examples of items in Life Environmental Studies

The following three examples are representatives of the three advantages of multimedia test which we described in chapter 1.

'Measure the objectives more correctly'

This item belongs to the unit of 'the Four Seasons'. It is important to judge by real environment in consideration of one's experiences. The teacher can find out whether the children can notice or not about how environments and lives around themselves change corresponding to the seasons.

The way of learning about change of the seasons is not reading textbook in learners' classroom, but going to the school grounds or a park, etc. Therefore this item enables to present the situations which are close to the scene of teaching in the lesson. Presenting of various situations leads to measure an ability of applying one's knowledge and experiences.

'Measure the objectives which had been difficult'

This item belongs to the unit of 'the Familiar Plants'. It is important that the passage of time is considered as understanding a biological matter. It is also important to understand correctly the place of changing with the passage of time.

The multimedia test enables to discriminate the passage of time and the place, the paper and pencil test doesn't. The passage of time is adapted to standard of evaluation. In other words, this is the item which testees make a integrative judgement from knowledge of different matters, for example knowledge of time and place.

'Create new objectives'

This item belongs to the unit of 'the Custom and Skills for the Daily Life'. It is important to greet others suitably in daily life. This item enables to measure an ability of problem solving with presenting with physical action. That is to say, testees have to recognise the given situation and context from nonverbal information because of using video in condition part. There is the only way of measuring in traditional test, as writing a greeting phrase. But inputting testee's own voice enables to measure suitability of the phrase (including intonation and volume of the voice) for the given situation and context. This educational objective is impossible to be measured in the traditional paper and pencil test. Therefore this educational objective is made revolutionary.

Figure 3

Figure 3: An example of a multimedia test item in Life Environmental Studies (1)

Figure 4

Figure 4: An example of a multimedia test item in Life Environmental Studies (2)

Figure 5

Figure 5: An example of a multimedia test item in Life Environmental Studies (3)

Closing

In this paper, we proposed to use multimedia in test, the characteristic of multimedia test is as follows:
  1. Multimedia test is suited for measuring knowledge application, problem solving and integrative understanding.

  2. In multimedia test, we can measure the educational objectives more correctly.

  3. In multimedia test, we can measure the educational objectives which had been difficult to measure.

  4. We can create new educational objectives that become to be measured only by multimedia test.
There are some problems for practical use of multimedia test yet, such as not spread hardware, lack of literacy for multimedia (an ability of operating multimedia apparatuses) and cheating. The hardware for multimedia will spread after several years. And the literacy will also spread. The way of testing which differs from the traditional way enables to prevent cheating. The adaptive test (each individual testee answers different items) and the individual test and so on are given as the examples. The group test is not required because of using computers and network. Therefore the multimedia test is effective for the evaluation in the distance education.

A multimedia raw material is often used in the plural number of items. Not the multimedia raw materials are put into the item bank but the multimedia raw materials are organised independently as the database with various information (for example the place and the date of recording, the authority, the related multimedia raw materials, etc). A multimedia database enables to assist the item creation and the test construction. After this, we must think about building multimedia database for test extended the way of thinking of item bank.

References

Chase, C. I. & Ludlow, H. G. (1966). Readings in educational and psychological measurement. Boston: Houghton Mifflin.

Ministry of Education (1989a). Course of study in elemental school. Tokyo: Printing Bureau, Finance Ministry.(in Japanese)

Ministry of Education (1989b). Guidebook for teaching in elemental school (Life Environmental Studies). Tokyo: Kyouiku shuppan. (in Japanese)

Linn, R. L. (1989). Educational measurement (third edition). New York: Macmillan Publishing.

Nagaoka, K. (1992). Micro Item Bank Test System: Interactive test construction, test execution, marking analysis and feedback. In Proceedings of Information Technology for Training and Education 1992, 471-475.

Author: Fujihara Yasuhiro, Bachelor, Graduate Student
Shibata Shimako, Student
Yoshikawa Yukari, Student
Dr Nagaoka Keizo, Associate Professor
3-11 Tsurukabuto, Nada-ku, Kobe 657 Japan.
Tel: +81 78 881 1212 ex.7080 Fax: +81 78 882 5959 Email: nagaoka@icluna.kobe-u.ac.jp

Please cite as: Fujihara, Y., Shibata, S., Yoshikawa, Y. and Nagaoka, K. (1994). Development of multimedia test system. In C. McBeath and R. Atkinson (Eds), Proceedings of the Second International Interactive Multimedia Symposium, 147-151. Perth, Western Australia, 23-28 January. Promaco Conventions. http://www.aset.org.au/confs/iims/1994/dg/fujihara.html

[ IIMS 94 contents ] [ IIMS Main ] [ ASET home ]
This URL: http://www.aset.org.au/confs/iims/1994/dg/fujihara.html
© 1994 Promaco Conventions. Reproduced by permission. Last revision: 4 Feb 2004. Editor: Roger Atkinson
Previous URL 16 May 2000 to 30 Sep 2002: http://cleo.murdoch.edu.au/gen/aset/confs/iims/94/dg/fujihara.html