Full Paper

Jan Les¹, Geoff Cumming¹, Neil Thomason², Sue Finch¹

J.Les@latrobe.edu.au, G.Cumming@latrobe.edu.au

1 - School of Psychological Science, La Trobe University

2 - Department of History and Philosophy of Science, University of Melbourne

Our starting point was the simple observation in the computer lab that it is natural and appealing to grab the mouse and make a short interaction with the learning environment while giving a spoken explanation. Students as well as teachers do this constantly. Would it be useful to give teacher and student a tool to capture such brief demos for later use by others? Our answer is Sam, as in 'Play it again...', which records and replays multimedia demos, with spoken commentary. Sam runs in the StatPlay learning environment for introductory statistics, but the Sam techniques could be used to build a demo facility in any application.

From a theoretical perspective, the problem is to provide guidance and explanation to a learner working in a learning environment. One approach to this problem is the Intelligent Tutoring System (ITS), which attempts to use artificial intelligence techniques to provide comments tailored for the individual learner. This approach raises problems of educational philosophy, and has not been very successful in practice. Even within the research field of Artificial Intelligence In Education (AIED), researchers are turning away from the ITS and are exploring a range of less onerous strategies to support the learner. One idea (Mayes & Neilson, 1996) is to make available to the learner a collection of excerpts from previous learner-teacher dialogues. Other proposed systems are based on stored expert answers to sets of questions (Ackerman & Malone, 1990; Graesser, Langston, & Lang, 1992).

Sam can store fragments of learner-teacher dialogues, or expert answers to questions, and so can be used to study these approaches to augmenting a learning environment. However, the demo is such a general concept that Sam can be used in many other ways as well. In this paper we first give a brief description of StatPlay and the current prototype Sam, then discuss some of the educational possibilities Sam offers. Key issues are the design of a database facility to organise for teacher and student a large number of Sam demos, and provision of branching and decision-making facilities to allow Sam demos to be linked flexibly into curriculum sequences.

StatPlay comprises demonstrations and interactive simulations to help learners overcome some fundamental misconceptions and acquire correct understanding of key concepts in statistics. The cognitive science rationale for StatPlay was described by Thomason, Cumming and Zangari (1995). By analogy with naive physics, we use the term 'naive statistics' for everyday beliefs about probability and statistics. An important goal in statistics education is to overcome the misconceptions of naive statistics. There is a useful analogy with the work of White (1993) and others who have shown how learning activities based on computer simulations can help learners overcome misconceptions in naive science. Partly influenced by that research, we have designed StatPlay to present vivid representations of the target statistical conceptions. We use a variety of types of learning activities, from guided to exploratory, and including games.

An important idea is that understanding requires the ability to express the target concept in several different ways, to translate readily between these different formulations, and to see how they relate. A pervasive goal in designing StatPlay and learning activities is to help students work with multiple, dynamically linked representations in order to build such understanding. Further descriptions of StatPlay were given in Cumming, Thomason, Howard, Les and Zangari (1995), Cumming and Thomason (1995), and Cumming, Thomason and Les (1997).

StatPlay is being developed in Visual C++ under Windows. The Windows 3.1 version of StatPlay has been used in increasing numbers of classrooms over four years and in 1997 was used by more than 3,000 students across more than 12 departments at The University of Melbourne. As a single example from the Windows 3.1 version, Figure 1 shows the screen display illustrating a surprising aspect of the Central Limit Theorem, which is readily discovered by exploration in the Sampling Playground, or microworld, of StatPlay. Even with a highly skewed distribution, here drawn freehand, and a sample size as small as n = 3, the sampling distribution of the mean shows surprisingly small skew: progression towards the Normal Distribution occurs surprisingly early as n increases.

We want a clean interface and excellent usability, as crucial features required if StatPlay is to be enduring and successful. Recent interface thinking has reached beyond cognitive psychology and human factors to become more diverse, with influences from theatre, various social sciences and post-modernism. The goal is to make the interface disappear: the user should be thinking about the task and the target concepts, ideally with no awareness of the tools. In moving from Windows 3.1 to Windows 95 we decided to redesign our screens from scratch and avoid off-the-shelf Microsoft components as being too visually complex. For our work on StatPlay and Sam, useful design ideas have included:

• Spare visuals that tell a story Tufte (1983, 1990) made a compelling argument for simple and clean statistical graphics. Tufte (1997) used wonderful examples to show that the best statistical graphics tell a story. We aspire to this level of presentation, and would like students to appreciate the power and clarity of well-chosen representations.
• User-centred design and successive prototyping Making repeated revisions in response to classroom feedback conflicts with the standard model of commercial software development that requires a full initial specification. It is refreshing that leading writers in human-computer interaction (HCI) are advocating the successive prototype approach that feels so necessary to us. Carroll's (1997) major review of HCI emphasises iterative design and the need to respond to users. He advocated 'throwing one away', as we are doing with our 3.1 version. Landauer (1995) built a persuasive case for user-centred design by giving examples of computer-related disasters that have resulted from other approaches.

These diverse and interesting writings do not give immediate prescriptions for the designer, but encourage us to think broadly about the whole statistical and learning purpose of StatPlay and Sam. They encourage us to talk to and observe our students with open minds, the better to build tools that help them work and learn effectively, and feel positive as they do so.

Figure 2 shows a prototype screen in the Distributions Playground of Windows 95 StatPlay. Figure 1 shows the familiar off-the-shelf Microsoft windows, buttons and other components. Figure 2, by contrast, shows the palettes, icons and other things that we have designed, following the principles mentioned above.

Sam allows the user to record and store as a demo any sequence of interactions with StatPlay, with a synchronised spoken commentary (Les, Thomason, & Cumming, 1997). Menu selection allows playback of any Sam demo.

To record, Sam stores the initial state of the whole StatPlay application-including the current screen-then stores microphone input of speech in real time, and mouse and keyboard inputs. To replay, Sam restores the initial state of the application, plays back the speech, and submits the stored sequence of mouse and keyboard signals to StatPlay at the correct times. The user sees and hears the same sequence of StatPlay behaviour, with synchronised spoken commentary, as was earlier recorded. Demos can be nested.

The key idea of Sam is that, at playback, submission of a suitable stream of inputs to StatPlay can reproduce the exact sequence of interactions that occurred during recording. To achieve this, the state of StatPlay at the start of playback must be reset to match exactly the state at the start of recording. Sam must be able to store and reset the state of StatPlay, and therefore Sam is application-specific. The great advantage of Sam is that only small amounts of information need be stored, and so demo files are small, little larger than the compressed digitised speech.

By contrast, Lotus ScreenCam allows the recording of demos with any software application, but is based on recording sequences of whole screen images. It therefore gives extremely large files even when sophisticated compression techniques are used.

Sam is implemented in Visual C++ 5.0. The fact that the Windows interface is message driven allows Sam when recording to use Windows hook functions to intercept mouse and keyboard input before this passes to the Windows kernel and then the StatPlay application. A copy of the input is time-stamped and stored in the demo file. At playback, the recorded initial state of the application is set up-the user sees the screen as needed for the start of the demo-then mouse and keyboard are disabled and Sam passes stored mouse and keyboard information at the proper time to StatPlay via the kernel. Digitised speech is played back synchronously.

A wide range of educational possibilities suggest themselves, including:

• Sam supports teacher-recorded explanations that are replayed at will by learners, for example when prompted by a printed worksheet. Sam demos can introduce topics, pose problems for learner solution, or give explanations on error.
• A set of demos can have a personality by being recorded in the language and style of a guide (Oren, Salomon, Kreitman, & Don, 1990), e.g. the expert professor, an anxious and painstaking student, or the well-known tutor. Alternatively a set could take a particular educational role, for example explaining the mathematical theory behind the concepts, or making the concepts relevant for economics, medicine, or some other particular discipline.
• It is easy to attach a demo to any icon or graphical feature on screen to give a brief explanation and demonstration, perhaps only a few seconds long, of the meaning or usage of the feature. Sam thus enables creation of a self-explaining interface, and an interface tutor. More generally, Sam can supply a part of the Help system.
• There are many puzzles, paradoxes and tricky examples that do not often arise in statistics textbooks yet can be revealing and educationally valuable. A set of demos of these could support a Lucky Dip, or offer the Strange Fact of the Day.
• Learners can make their own demos, and submit queries in demo form for tutor comment. Learners could be required, singly or in small groups, to make their own demo to explain a concept. As well as being a useful learning activity, such a demo, submitted for assessment, would be a sensitive indicator of learner understanding.
• A demo can be a record of a teaching-learning interaction involving more than one person. A particularly useful question and answer, or fragment of negotiation, could be restaged for recording as a demo.
• Extended Sam recordings could be used to store for research purposes a detailed record of learner and teacher interaction with StatPlay. The voice channel could record a learner's think-aloud protocol, rather than spoken instructional dialogue.

We have used the Sam prototype with students in three psychology and environmental science statistics subjects. It has proved quick and easy for teachers to record demos of very acceptable quality. Students have worked in pairs or small groups, with printed worksheets to guide their activities, including suggestion of which Sam demos would be useful. Demos were presented via loudspeaker or multiple headphones. In every case there was an immediate positive response, with many teachers and students enthusiastic about the appeal, ease of use, and educational value of Sam demos. Demos were also seen as making it easy to become familiar with the interface and the range of features offered by StatPlay.

Almost the only questioning comment was from one or two tutors who noted that sometimes their students were less available for talk with the tutor because they were absorbed with the spoken message from StatPlay!

Our Sam prototype supports only a simple menu list of demo titles, which is unwieldy beyond one or two dozen items. The next version should support many more demos, but we need to keep strict limits on complexity and development time. There are two central challenges:

• We would like to offer a vast number and range of demos, yet make it very easy for the teacher or student to find the best one for now.
• The teacher or student should have freedom to explore, but we should also be able to construct curriculum sequences that are very easy to use in various ways.

Some demos serve as part of the Help system, and should be linked to the icons, statistical symbols or other display elements they explain. Other special purpose demos provide explanations of statistical terms, and are accessed via a glossary list.

Most demos are intended to contribute to statistics learning activities. Information can be stored automatically with each demo, including author, date of creation, StatPlay playground, and duration. The author can add descriptors, including demo title, subject context and topic, and a brief free text summary of the demo. A relational database is then the natural structure for demo storage, so that a user can scan and select a demo for replay via any of the types of descriptors, or via a text search of demo summaries.

When a user is working in StatPlay and calls up Sam, presentation of the demo database should be as an easily-traversed tree. The presentation should be sensitive to the most recently used demo, and to the current location in StatPlay. In other words it should be easy to select demos close to the one most recently used, and most relevant to the current context in StatPlay.

When scanning to select a demo for replay, the user should see the descriptive information and a small preview picture showing the screen at the start of the demo: in many cases this picture would give a fast and natural indication of demo content.

We are convinced that it is best to keep demos short, usually no more than a minute, and to have students spend most time carrying out their own activities. Therefore we need to be able to specify sequences of related demos. A teacher should, for example, be able to arrange for a student to play one demo, do some StatPlay work, play the next demo, maybe elect to step sideways to a demo giving further explanation, do some more StatPlay exploration, decide on the answer to a question, be directed to a demo appropriate to the answer given, and so on. We need to offer the teacher easy ways to set up links, and choice points so that a curriculum structure can be specified, drawing on a set of related demos.

There are many exciting possibilities, including the development of demos into agents with some limited ability to converse with the user. For now we need to select for construction a very limited number of facilities so that the Sam interface is simple and the software development task is manageable. Our current plan is to provide the creator of a demo with the following facilities:

• A text window appears at the end of demo replay, with buttons linked to specified further demos. Text could pose a question, and answer options could lead to different follow-on demos. The user could have full use of StatPlay before choosing to answer the question.
• Choice whether at the end of a demo StatPlay reverts to its state at the start of the demo.
• Any sequence of demos can be named, described and entered into the demo database, which thus becomes a database of single demos, learning sequences of related demos, and even curriculum parts built from such sequences.

A learner working through a sequence would at any time be able to see the current position in the sequence, to step back or forward, or to leave the sequence.

The naturalness and appeal of our demos within StatPlay and the immediate positive response of users to Sam convinces us that the multimedia demo has enormous potential. The key technical issue in providing Sam facilities in other applications is the need to store the full state of the application at the start of a demo. There are software library facilities now becoming available that make it easy for an application to be designed so that it can 'store its own state'. It is therefore feasible to think of Sam being incorporated into a wide range of software. In most cases Sam demos could form the most user-friendly part of the Help system, and could also be arranged to give tutorial sequences to assist new users.

When the application has, like StatPlay, education as its primary purpose, demos can play an even more central role by offering a wide range of support to the learner. Many pedagogic and implementation issues need study before the full educational potential of the demo is realised.

StatPlay has been supported by the Committee for the Advancement of University Teaching, the Committee for University Teaching and Staff Development, the Australian Research Council, the Apple University Development Fund, and The University of Melbourne.

(c) Jan Les, Geoff Cumming, Neil Thomason, Sue Finch

The author(s) assign to ASCILITE and educational and non-profit institutions a non-exclusive licence to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. The author(s) also grant a non-exclusive licence to ASCILITE to publish this document in full on the World Wide Web and on CD-ROM and in printed form with the ASCILITE 97 conference papers, and for the documents to be published on mirrors on the World Wide Web. Any other usage is prohibited without the express permission of the authors.