ASET logo
[ EdTech'96 Contents ] [ EdTech Confs ]

Automatisation of student assessment using multimedia technology

David Taniar
Swinburne University of Technology

Wenny Rahayu
LaTrobe University

Most use of multimedia technology in teaching and learning to date has emphasised the teaching aspect only. An application of multimedia in examinations has been neglected. In this paper, we present a prototype of multimedia question bank, which is able to handle questions consisting not only text but also images, video recordings, etc. We also describe retrieval techniques, and show some sample results.

Introduction

As multimedia technology has become more available, the benefits of using this technology in teaching and learning become more visible. Multimedia is often referred to as a combination of different medium (eg, images, graphs, audio, video) used in a software application (Christodotilakis and Koveos, 1995, Grosky, 1994). Incorporating these media in computer based tutorials (CBT), one of the devices in educational technology, results in a much more flexible and interesting learning process. However, most of these tutorials concentrate on the delivery of the lesson only. Assessment, which is another part of teaching process, is often neglected. The use of multimedia technology in this area has not been considered in depth. In this paper, we would like to address how multimedia technology can he applied to the automatisation of assessment.

One issue in automatisation of student assessment is the availability of question banks. Multimedia question banks typically consist of questions with multimedia features. This means that the questions can be presented not only in text, but also in any other forms, eg, images, video. Furthermore, some questions can be grouped and they refer to a common object (ie, image, video). For example, in the database there are ten questions which are based on the same video recording. A major problem arises when only a few of these questions are selected. A decision must be made whether to present the object as a whole or only parts of the object which directly relate to the questions. If the object is large, such as a video recording of 30 minutes, and only two or three questions are selected from this recording, it would be wiser if only relevant segments of the recording are played. Presenting the whole recording will not only distract, but also confuse examinees.

Choosing which part(s) of the object to accompany the selected questions has become a non-trivial task. Our approach is to use the object oriented technology which provides a feature of message passing from one object to another. In this way, the selected questions only need to send a message to the object, and let the relevant object partition and construct the segments to be presented together with the questions. This technique works because an object is stored as an aggregate of its components, not as an unbreakable unit. Our previous work (Taniar and Rahayu, 1995) discussed the storage and retrieval techniques of multimedia questions in great detail.

In this paper, we would like to present a prototype of a multimedia question bank. The rest of this paper is organised as follows. Section 2 describes the structure of multimedia questions. Section explains how to retrieve questions from the database. Section 4 presents some sample results, and finally, Section 5 gives a conclusion.

The structure of multimedia questions

The two main parts of multimedia questions are object and question parts. Figure 1 shows a typical multimedia question. In the context of object orientation, "object part" should not be confused with "object". Object part contains a "thing", which can be of any form, ie., image, graphs, recording, text (Taniar and Rahayu, 1995). In contrast, the term "object" in the object oriented paradigm refers to an instance of a class (Dillon and Tan, 1993, Taniar, 1992).

Figure 1

Figure 1: Multimedia questions

Object part

The object is the main focus to which questions in the question part refer. These objects can be either: Temporal objects are objects that span a linear dimension. An example of a temporal object is a recording of any type, such as audio, visual. These objects fill a single dimensional space which is the duration of the play. A key feature of this kind of object is that there is a starting point and an ending point.

Spatial objects are static. They can be presented on x and y axis, or columns and rows. Some examples include images, graphics, text, etc. When the object does not fit into the provided space, a scroll bar is shown to enable users to scroll the object.

Question part

The question part consists of a set of questions referring to an object. The questions can be either: A single point question is a question which has a reference to a single segment of an object. If the object is a temporal object, single point questions relate to a "moment" of the play. The "moment" can be interpreted as a section that cannot be divided into more sections. The question could be a question on what was said, or where it took place, etc. If the object is a spatial object, single point questions refer to a section of an image/picture. For example, a question can be asked about a part of an engine, or a block of a landscape.

Multi point questions have reference to several sections of an object: The referred sections can be in serial or at random. Serial multi point questions associate with a series of adjacent sections. If it is in a temporal object, the sections are contiguous. However, in a spatial object, adjacent sections can be clustered based on columns or rows. On the other hand, random multi point questions link with several sections of an object in random order. In an extreme case, a question refers to the starting point and ending point of a recording. Therefore, when this question is selected, all parts of the recording must be played. This similar technique applies to spatial objects as well.

Some examples

Figure 2 shows an example of questions with a spatial object. The object is a music score of Beethoven Symphony No. 5. Since the music score is very long, a scroll bar is provided at the bottom. The questions related to this object can be viewed by pressing the "Question" button.

Figure 2

Figure 2: Spatial object - music score

If the music is played, rather than displayed, the object becomes a temporal object. Figure 3 shows an example of a temporal object. The object itself is a video recording of an orchestra playing Beethoven Symphony No. 5. It is not possible to represent temporal objects on paper. Rather a moment of the recording is shown, where the orchestra is in action. The questions, which will be viewed by clicking the "Question" button, are suitable to test the listening comprehension of students studying at a conservatorium of music.

Figure 3

Figure 3: Temporal object - music recording

Retrieval techniques

Question retrieval should be two directional, ie. from the object parts or from the question parts. As a result, searching techniques should facilitate both object based and question based retrieval.

Object based retrieval

Content based searching of multimedia objects is considered immature as current video and audio management tools are based on pixel rather than the perceived contents (Smoliar and Zhang, 1994). Current research in the areas of image and pattern recognition is addressing this problem, and therefore is excluded from our implementation.

In our prototype, each object part has a set of properties, ie., title, topic. Users are able to do a search based on information contained in these properties. Figure 4 shows an example of question retrieval based on objects. In this example, the user is able to search an object on a particular title. Once the object is found, all related questions will also be retrieved.

Figure 4

Figure 4: Property based object retrieval

If the number of objects in the database is small, searching can be done through a complete browsing of all objects. Once the objects are selected, the associated questions are then presented and chosen, as associated questions have been implemented as an aggregate of the object parts. Therefore, each object has a number of corresponding questions. Figure 5 shows a number of objects to be manually selected by double clicking a desired object. The title of the object itself appears at the top of each object.

Figure 5

Figure 5: Object browsing

Question based retrieval

Selecting questions from a database can be done through a query facility within the database, which retrieves questions satisfying the selection criteria. This selection can be done through keywords or subject matching. At this stage of the implementation, content based searching for questions is not possible. Figure 6 shows a property based question retrieval. The user has to enter the subject of questions to be retrieved.

Figure 6

Figure 6: Property based question retrieval

Figure 7

Figure 7: Questions browsing

Another retrieval technique is based on database browsing. Users are presented with a complete list of questions in the database. The user can select a particular topic, and then within each topic, the questions can further be selected and chosen. This technique will be very useful when combined with the object based retrieval, as the object based searching lists an object. Based on the selected object, a number of questions can be chosen either manually or randomly. Figure 7 shows an example of question browsing retrieval technique. Once the questions are selected, the accompanying objects are also retrieved. This mechanism is possible because each question is attached to an object.

Sample results

Some retrieval results are presented as follows. Figure 8 shows an example of the result of multi point serial questions. The object presented shows a string part only (not a full orchestral part). Question 1 refers to the tempo of the music. The tempo is usually displayed at the beginning of a section. As the section concerned is the first movement of the symphony, only the very first few bars are displayed. Question 2 refers to the clef used by a particular instrument (ie., viola). Combining questions 1 and 2 only the first few bars of the symphony, which also include the viola part, are displayed.

Figure 8

Figure 8: Multi point serial questions

Figure 9 shows an example of multi point random questions. Question 3 refers to the instruments playing on the few first bars on the symphony (in this case, they are Clarinet, Violin I and 2, Viola, Cello, and Double Bass). Question 4 refers to the Clarinet and Violin I sections. Both questions 3 and 4 are multi point random questions, because they refer to two or more different types of instrument which are not adjacent.

Figure 9

Figure 9: Multi point random questions

Conclusion

A prototype of a multimedia question bank has been presented in this paper. The type of question that we concentrate on is an object referred to by several questions. Retrieving some of the questions can be very tricky because not all parts of the objects are concerned. Our system is able to select parts of the object which must be presented with the questions.

Future work includes finalising technical details of the system. It is also planned to have a useability testing of the system when available.

References

Christodoulakis, S, and Koveos, L. (1995). Multimedia Information Systems: Issues and Approaches. In Modern Database Systems: The Object Model, Interoperability, and Beyond, W. Kim (Ed), Addison-Wesley, ACM Press.

Dillon, T., and Tan, P-L. (1993). Object Oriented Conceptual Modelling, Prentice Hall of Australia Pty Ltd.

Gemmell, D. J, Vin, H. M., Kandlur, D. D., Rangan, P. V, and Rowe, L. A. (1995). Multimedia Storage Servers: A Tutorial. IEEE Computer, May 1995, IEEE Computer Society.

Grosky, W. I. (1994). Multimedia Information Systems. IEEE Multimedia, Spring 1994, IEEE Computer Society.

Rahayu, J. W, and Chang, E. (1993). A Methodology for Transforming an Object Oriented Data Model to a Relational Model. Proceedings of the Twelfth International Conference TOOLS Pacific, Melbourne.

Rahayu, W., Chang, E., and Dillon, T. S. (1995a). A Methodology for the Design of Relational Databases from Object Oriented Conceptual Models Incorporating Collection Types. Proceedings of the TOOLS18 Pacific, Melbourne.

Rahayu, W., Chang, E., and Dillon, T. S. (1995b). Transformation of Object Oriented Collection Types into Relational Logical Models. Industrial and Poster Paper Proceedings of the 14th International Conference on Object Oriented and Entity Relationship Modelling, Gold Coast.

Smoliar, S. W., and Zhang, H-J. (1994). Content Based Video Indexing and Retrieval. IEEE Multimedia, Summer 1994, IEEE Computer Society.

Taniar, D. (1992). Unifying Object Oriented Features with Database Technology: A Survey, Masters Thesis, Swinburne University of Technology, Melbourne.

Taniar, D., and Rahayu, W. (1995). Multimedia Question Banks: Storage and Retrieval Techniques. Proceedings of the Twelfth Annual Conference of the Australian Society for Computers in Learning in Tertiary Education, ASCILITE'95, Melbourne.

Authors: David Taniar
Department of Information Studies
Swinburne University of Technology
Hawthorn, Victoria, Australia
taniar@matilda.vut.edu.au

Wenny Rahayu
School of Computer Science and Computer Engineering
LaTrobe University
Bundoora, Victoria, Australia
wenny@latcs1.lat.oz.au

Please cite as: Taniar, D. and Rahayu, W. (1996). Automatisation of student assessment using multimedia technology. In J. G. Hedberg, J. Steele and S. McNamara (eds), Learning Technologies: Prospects and Pathways, 159-167. Selected papers from EdTech'96. Canberra: AJET Publications. http://www.aset.org.au/confs/edtech96/taniar.html

[ EdTech'96 contents ] [ EdTech Confs ] [ ASET home ]
This URL: http://www.aset.org.au/confs/edtech96/taniar.html
Last revised 7 May 2003. HTML editor: Roger Atkinson
Previous URL 12 Mar 1999 to 30 Sep 2002: http://cleo.murdoch.edu.au/aset/confs/edtech96/taniar.html