IIMS 96 contents
[ IIMS 96 contents ]

Your mentor on the superhighway: Just in time multimedia training for corporate and government

Michael Ross and John Merakovsky
Computer Power Education
Online multimedia training delivery systems provide a framework and a structure which is revolutionising the way training can be conducted in business and corporate environments. This paper examines some of the new education and training models that will be available with the emerging online multimedia training systems. The first half of the paper describes the Advance system which is currently being developed by Computer Power Education. Advance is an online multimedia training delivery system designed for corporate and government clients. Its features and benefits for corporate training are reviewed, followed by a brief discussion of the new educational models that are possible with Advance. The second half of the paper presents a progress report of findings from the educational evaluation of the prototype system conducted to date. The models trialed included a virtual classroom, online tutor support from remote locations of students carrying out self paced training, and a multimedia enriched live classroom. The paper concludes with a brief summary of design implications for multimedia content delivered via a network and used for corporate training.


Introduction

In 1993 Computer Power Education launched a major research and development initiative to develop an online multimedia training delivery system. The system, known as Advance, will be completed for commercial release in late 1996. The aim of the Advance project is to integrate the most advanced telecommunications and computing technologies with educational and training methodologies to build a system which can deliver customised, seductive training where, when and how it is needed most. The project has funding of approximately $7.5 million, and currently employs some 25 full time people, making it perhaps the largest multimedia training R&D project in Australia. The project is concerned with the development of both the technology for delivery of multimedia training and the human factors associated with the design and delivery of multimedia training. The outcomes of this project will provide the means to deliver a new vision of workplace training which will greatly improve the productivity and international competitiveness of business.

Computer Power Education (CPE)

Computer Power Education consists of two education and training businesses: Management Technology Education (MTE) and the Computer Power Training Institute (CPTI) and has posted record growth in recent years. MTE and CPTI have training locations throughout Australia and in Hong Kong.

Computer Power Training Institute (CPTI)
For over twenty years, CPTI (formerly the Control Data Institute) has been dedicated to teaching computer and business skills. It has institutes in all capital cities of Australia. The CPTI training courses are entirely self paced and each institute provides full time tutor support to assist the students. Its courses have been developed in conjunction with industry, government and universities to provide relevant, well recognised qualifications. At any one time CPTI has over 1700 students enrolled in its institutes throughout the country.

Management Technology Education (MTE)
MTE was established in 1982 and today is the leading provider of education to the business and government sectors in Australia with a comprehensive range of training courses. The areas of activity include: public seminars - on management and technology issues, hands on PC training Australia wide, management skills training, and customised courses. Most of their courses are two days but range from one to three days in duration. MTE has over 100 instructors and provides over 30,000 training days per annum. More than 400 of Australia's top 500 companies use MTE's services.

What is Advance?

Computer Power Education recognised that the corporate and government training markets were changing. New educational methodologies and new technology were revolutionising the way work place training could be conducted. In order to protect their market leadership in these industries they initiated the Advance project. The Advance system is an education and training system which integrates the most advanced technology, software, and human factors methodologies into a coherent training system.

The technology of Advance
The technology refers to the client/server architecture the networks and telecommunications equipment required to deliver multimedia to the desktop. Customised software is being developed to integrate all the functions required to enable a learner to locate the desired training, see a preview of it, download it to their PC, and carry out the training. Or, if the learner is participating in a virtual classroom or online tutorial, to enrol in that course. The diagram below represents a schematic diagram of the Advance system.

Figure 1

Figure 1: Schematic diagram of online delivery system

The human factors of Advance
Human factors is concerned with the way people interact with technology. The study of human factors in the Advance project has a dual role: first, to apply scientific research methods to the design and development of the Advance system so that it matches needs of users; and second, to design the training methodologies and change methodologies which can be used to implement the Advance system into corporations.

Just in time multimedia training

In this section a brief description of some of the basic concepts of the Advance system are presented.

What is just in time training?

Just in time training is delivered to the learner as soon as possible after a learner has identified a specific training need, and the shortest possible time before a learner needs to use the skill and knowledge outcomes of the training. The diagram below represents the important features of JITT.

Figure 2

Figure 2: Schematic model of just in time training

JITT is concerned with getting the right training to the learner when, where and how they want it. The main feature is that the training is appropriate and delivered in a timely manner. The rapid development of educational computing and telecommunications technology is quickly increasing our ability to deliver just in time training. The wealth of training content and courses around the world will soon be able to be accessed much more easily through the use of this technology. Multimedia content can be delivered to remote locations via an online service, CD-ROMs, or diskettes. Video, audio and data conferencing can be used to deliver training to learners in remote locations. Online tutors and helpdesk service can be used to provide answers to questions and enquires. These are some examples of how technology can be used to provide just in time training.

The availability of telecommunications systems with high bandwidth and networked computer system allows for a number of just in time training models. The major ones are described below.

Educational models of just in time training: "The Bank" scenario

To help explain how online training services can be used, this section is based around a fictional scenario of a major national bank. "The Bank" has approximately 20,000 staff in Australia and overseas, and 1600 branches. Its head office is in Melbourne where there are 1500 management and administrative staff.

The Advance system is designed to allow multimedia training to be delivered via an online service to the desktop. This service facilitates the training at any time or any place. For example, the training manager of The Bank is required to train 7000 managers in four major areas: product knowledge, management skills, PC skills, and legal & ethical issues. Each new manager requires three weeks full time induction training and five days per annum thereafter. There is a 30% turnover of staff. The Bank provides approximately 56,000 training days per annum for the managers alone. The Bank estimates it costs $400 per day, per person to provide management level training, leading to an annual training cost of approximately $22.4 million.

The use of CBT has been demonstrated to improve the efficiency and cost effectiveness of training in many different corporate environments (Advance Project Report No. 3, 1995). The Bank decided to convert their training from predominantly instructor led classroom based to multimedia training using combinations of multimedia CBT and data, video and audio conferencing. They designed their multimedia training to be self contained, coherent chunks which could be selected individually by the learner to build a customised training program. The training varied in sophistication from online documentation to full multimedia with advanced instructional design and multiple learning pathways. Also available to the managers through the Advance system were virtual classrooms and online tutors using full conferencing (audio, video & data) services.

The following section describes how the Advance online training services can support The Bank's training scenario.

Multimedia computer based training
With the development of new products in The Bank it became policy to develop multimedia CBT to be released with the new product. The CBT is then downloaded to the branches and each manager completes the training without ever having to leave the branch office. Each module is competency based and contains assessment exercises. The results of the assessments are relayed back to head office to be scored and recorded on the manager's personal records. If the assessment process requires a more interactive personal assessment, the video conference facilities are used. If, for example, the product is a new home loan package and the manager has to demonstrate selling the product to a customer, that assessment can be conducted without the manager or the assessor leaving their respective offices simply by conducting a role play with the video conferencing facilities built into Advance. Another advantage of designing multimedia CBT for online delivery is that the training package can be amended quickly and easily if there are any changes by simply downloading the altered file to all the users, rather than having to press a new CD-ROM or send out new diskettes.

Aspects of PC, management and the legal/ethical training can be conducted using multimedia CBT as well. The Bank uses generic PC skills and management skills CBT which is available on the Advance database for the managers to use at anytime they want to. The Bank has also developed a simulation game to train their managers in the legal and ethical aspects of the role. While all of these training courses can be conducted entirely independently of the instructor there are some aspects of management training that require an instructors assistance. The Bank estimates that 60% of their training can be conducted through multimedia CBT.

Online tutors
The Advance system has a facility where learners can call an online tutor to request information or tuition. The Bank has a series of product experts around the country who have made themselves available to act as online tutors. They are rostered on the Advance database and can be called upon during their rostered period. If a manager located at the Derby branch is with a customer who has a question about tax implications of business loan product, then the subject matter expert who happens to be located in Sydney can be contacted in real time to provide the required information. Or, if a manager is in the middle developing an annual budget and requires assistance with Excel 5 to format a graph in a particular way, then that manager can connect to an online tutor service provided by a commercial training company such as MTE. The tutor can take over the manager's screen and demonstrate the required solution through application sharing and explain what is going on using the audio link. This type of service is akin to what the help desk and electronic performance support systems are trying to achieve. The Advance system will support this kind of just in time training which is not currently being met with today's training technology.

Instructor led training with online multimedia content
The managers have been complaining that they keep being interrupted every time they try to carry out their training in the workplace. They require 15 hours training in "Word 6", but do not have the time to spend two days at a training course in the nearest capital city. Another alternative with the Advance system is to use multimedia CBT to obtain the basic skills related to the functionality and operation of the software and then attend a live class for half the amount of time (one day) to cover the more specific problems such as handling long documents. The manager could enrol in an MTE multimedia class, receive the pre-course multimedia CBT which is completed in the office, and then go to the physical class. The physical class can be much more intensive and cover the specific areas that are required by the manager. This allows the efficiency of the training session to be greatly increased as the learner enters the one day training with a much higher base level skill than would be normal on a two day course. The model would benefit people who prefer to have some level of instructor led training in the classroom, but cannot afford much time out of the office.

Virtual classrooms
The final model to be discussed here for The Bank managers is that of the virtual classroom. The Bank requires that all their managers understand the basic principles of capital gains tax. To achieve this they decide to run as series of virtual classes led by the training department in Melbourne. Managers from all over the country book into the two hour virtual class at a time that suits them. The booking, conduct and evaluation of learning can all be done through Advance. The managers simply have to be at their desk and logged in at the time the course is due to start.

The course is led by an instructor, and the communication is facilitated by audio, video and data conferencing software which links all the participants. The instructor presents the concepts, sets tasks for the participants, promotes discussion, and generally facilitates the learning of all the participants. Participants can respond, see each others' screens to view slides or presentations, and see each other in order to enhance the quality of communications. The main benefits of multimedia online training delivered in this manner are the huge savings that can achieved in both time and cost by allowing learners to undertake their training from the workplace.

Educational evaluation of Advance

The development of the Advance system has been driven from a user centred process. One of the central tenets of this perspective is to ensure information from the users is systematically collected and presented to the designers and developers in a form that can be incorporated into their work. The Advance project has included many user evaluations and tests. The major ones are reported below.

The emphasis of a user centred is to ensure that the design and development process is iterative, formative, and situated. The final system grows into a useable product through an iterative process to design, develop, test and re-design, re-develop and re-test. Feedback from users is sought and given to the designers and programmers on a regular basis. Additionally the testing is conducted in the real environment that the product is going to be used in, and it is conducted carrying out the activities that the final product is going to be used for. In this way the requirements of the users are always the driving force for the development activities.

The diagram below represents the process.

Figure 3

Figure 3: The major educational trials planned and conducted
during the design and development of the Advance system

Figure 4

Figure 4: Model of evaluation for the prototype Advance training system

The diagram above identifies the major aspects of the test and evaluation activities conducted in the development and trialing of the Advance system. During the design and development activities, unit and integration testing, useability testing and process testing were conducted.

February '95: Evaluation of the multimedia classroom

The aim of these studies was to evaluate the effect of using multimedia in two of MTE's most popular courses. Four trials of Introduction to Excel 5 and two trials of Management Skills for Supervisors and New Managers were conducted. The courses were redesigned to include a number of educational changes: the use of multimedia in the classroom, pre-course computer aided instruction, a pre-course student questionnaire which measured student approaches to learning and background skills and knowledge, and delivery of the course in a one day format. Fifty eight participants selected from twenty major Australian companies agreed to take part in the trial classrooms. Participants volunteered to attend one of the six trials. Observers collected evaluation information during the trials, each participant filled out an evaluation questionnaire on completion of the day's training and were contacted for a telephone interview three weeks after the course. Each instructor was also interviewed at the end of his or her training course.

The results indicate that both the participants and the instructors were very positive towards the use of multimedia in the MTE classroom. They felt that they had learnt new skills which could be applied in the workplace. They also reported that they would be comfortable using multimedia in the workplace. The use of pre-course computer based training (CBT) was reported to have improved the effectiveness of the day's classroom training, and participants viewed the CBT as a valuable learning tool. The results from using a pre-course questionnaire were inconclusive. The instructors reported that they did not have enough information to be able to make any significant changes to their presentations on the basis of the questionnaire results. The delivery of the Excel 5 course in one day worked well. Instructors had no trouble covering the curriculum in that period. The participants and instructors reported that there was too much material in the Management Skills course for one day. (Advance Project: Evaluation of the MTE Multimedia Classroom: February 1995).

September '95: Evaluation of online delivery of multimedia training

Having demonstrated the efficacy and user acceptance of the live multimedia classroom, in the February '95 trials, significant systems development took place to build a prototype of the Advance system. The prototype was to designed to be able to support the basic activities required to deliver online multimedia training. A series of application effectiveness tests were conducted to evaluate the useability of the system prior to its utilisation for training. After the useability testing four educational trials were planned and conducted. The first two were repeats of the basic process for delivery of multimedia to a live classroom (as per February trials), the major difference being that the classroom was networked and the multimedia was delivered online to the students' PCs. The third trial evaluated the concept of being able to deliver self paced training to CPTI students who were supported by an online tutor. The final trial evaluated the virtual classroom model where students were led through a training program by an instructor in a remote location to the students and all the students were in remote locations. The evaluation objectives and methods are described separately, and the major findings and outcomes are summarised below.

Applications effectiveness and process testing
Applications effectiveness testing was performed in order to assess the Advance design in terms of efficiency, functionality and ease of use from a user's perspective. Results from useability testing are used in the iterative development cycle to improve the performance and functionality of Advance.

Test objectives
The objectives of useability and process testing were to ensure that Advance meets certain standards with respect to system useability. The criteria that were examined in the useability testing are: visual clarity, consistency, compatibility, informative feedback, explicitness, appropriate functionality, flexibility and control, error prevention and correction, user guidance and support, system useability problems, general satisfaction, learnability, and retention.

The objectives of process testing were to ensure that the overall process (from login to course completion)

Test components
The useability testing can be classified into the following components Method
The process testing was based on detailed test plans, which were developed after walk throughs on each component with the system team. Samples of representative users were invited from corporate client to participate. Eight people took part in the testing, their results recorded and reported in the Advance Applications Effectiveness Test Report (1995). Detailed findings are not reported here as they strongly related to the contextual aspects of the Advance software. A summary is presented at the end of this section.

Multimedia enhanced live classroom

Two multimedia enhanced classrooms were conducted: the first "Introduction to Excel 5", and the second "Leading Teams". These courses were representative of the most popular short courses offered by MTE. They are also quite different in style. The former being knowledge based and designed and designed to enhance learners skills in a widely used PC application. "Leading Teams" is designed to build the learner's interpersonal and leadership skills; focusing on communication, team building, motivation, and personal styles. Information collected about the application of multimedia was hypothesised to be quite different for two courses.

Objectives
The following objectives were identified for the trials.

Method
Six learners participated in each of the two multimedia classroom trials. The learners were selected from a group of potential corporate users. The corporations were invited to select participants on the basis that the participant required or wanted training in either "Excel 5" or team development/ leadership. The diagram below identifies the major steps in these two educational trials.

Figure 5

Figure 5: Model of prototype multimedia enhanced classroom

Online Tutor/Open Access Centre

This model was developed to support CPTI's particular training requirements. The course materials are entirely self paced and consist of paper based workbooks and diskettes with established example files. Two course selected for this trial: introductory and advanced "Excel 5".

Objectives

Method
Three students were selected for the introductory Excel course and three for the advanced course. The students were full time, fee paying students of CPTI, and were the first three students who were ready to start their respective course at the time the trials were due to be conducted. Students were given an hour's briefing on the how to use the Advance system, but apart from that they were treated in exactly the same way as they would have been if they were doing the course in the normal manner. The course usually takes and average of four days, with an assessment on the fifth day. This may vary from case to case as the course is self paced and therefore dependent on the individual. After the formal assessment students filled in an online evaluation questionnaire. The diagram below shows a student receiving online tuition.

Figure 6

Figure 6: Student receiving online tutorial

Virtual classroom

The virtual classroom describes a training environment where the learners and instructor are all at different locations, and instruction is facilitated by a technology based communication system. In this trial four people participated and the course was "Introduction to Excel 5". The content of this course was the same as for the multimedia enhanced classroom trial, but all of the participants were different (except for the instructor).

Figure 7

Figure 7: Virtual classroom instructor

Objectives

Method
Participants were invited from major Australian corporations on the basis that they had a need to be trained in "Excel 5". None of the participant had taken part in any of the previous trials, nor had they been trained in "Excel 5" before. Students carried out pre-course multimedia training prior to coming to the day's training (as with the live classroom). During the pre-course training students were also briefed on how to use the Advance system. Training was conducted via audio and data links with the instructor demonstrating and leading the day training. Online multimedia was available and an online evaluation questionnaire was completed at the conclusion of the training. All participants were observed during the day by a dedicated observer and interviews were conducted at the conclusion of the training session.

Summary

The results of the first extensive tests of the Advance prototype support the position and direction that the ITS project has taken.

Useability testing demonstrated that the Advance applications are highly useable, and confirmed that integration between components such as multimedia and conferencing is a key element in the educational effectiveness of such training technology. The learner response to multimedia instruction was very positive, particularly in the context of the instructor led classroom. When supplemented with pre-course training, there is strong evidence to support the educational viability of a one day multimedia enhanced classroom. The Open Access Centre/Remote Tutor trial was extremely successful. The students found the learning centre to be very focussed and supportive of their educational needs. Remote instruction need not require video conferencing and broadband services, which means that this model is viable as a business proposition today. Of course, a certain base level of technology is required, but the design of the courseware and online techniques of the tutor are central to the success of this model.

The virtual classroom represented the most challenging of the trials, both from an educational and technological perspective. The model was demonstrated as an effective means of delivering synchronous training to multiple, geographically isolated learners. The use of multimedia in the virtual classroom provided both a richness to the class and a break from the intensity of the instructor led training.

From the instructor's perspective, the virtual classroom is challenging and requires greater effort to maintain concentration. This will be alleviated in part by improving the suability of the system. Again, system integration and sound planning are key elements in removing impediments from the educational process.

As a results of the testing and trials, recommendations have been made for the development of the Advance system in the next phase. In addition, the project has developed considerable knowledge in course and content development for online delivery, and the skill set required of the online instructor. This information will be used in the development of a Course and Content Developers Guide during the next phase.

The Advance prototype trials have demonstrated that online instruction and multimedia enhanced classrooms can be run successfully, with today's technology and a sound understanding of educational processes.

Conclusions

Development of just in time training

Just in time training for business and industry will become increasingly prevalent with the advent of online system and multimedia technology. For the systems to reach their potential a great deal of development work is required to build the services that will be delivered via the technology.

Online delivery of multimedia content

Delivery of online multimedia will require two primary developments: a system of searching, locating and downloading multimedia training units quickly and accurately, and the development of content which is modular - both educationally and technically - and based on what people are actually required to do in the workplace. Competency based curriculum design guidelines provide a good model for the instructional design of future online training resources. Training units will have to be cohesive, self contained and able to be combined in multiple ways to form an appropriate training course. They will be short - the ideal length of time being approximately 20 minutes. This allows the users to quickly acquire the unit of competency and proceed with their task with minimal disruption to their work resulting in large gains in productivity.

The instructional design will allow learners to identify the precise outcomes of the training, and to conduct a self assessment on completion of the module. The training content does not have to be sophisticated in its use of multimedia, but it does have to relevant and appropriate to the training needs. In this way it will become engaging both because of the instructional design, and because the environment in which it is learned. The data files should be structured in such a way as to allow fast transmission through telecommunication networks with limited bandwidth. A learner should be able to begin training even if later sections course are still being downloaded in the background.

Online tutors

The major benefit of integrating online multimedia technology is the flexibility that can be achieved for instruction. Courses may range from entirely self paced CBT to fully controlled instructor led training. Training providers will need to analyse their current mode of delivery and determine how their courses can be converted to take advantage of the new delivery systems. Online tutors will have to be rostered on to be available if real time support is required. They will also have to be trained how conduct their tutorials using the online training technology.

Virtual classroom

Course providers face similar requirement with virtual classroom development as they do with online tutors. instructors will have to be much more sensitive to the potential difficulties a student might be having than in a face to face classroom. This is because of the greater degree of difficulty instructors have in getting the normal type feedback they would receive in a live classroom. Careful planning must go into how a course is going to be delivered, what techniques are likely to work, how is the assessment managed, and many other instructional design questions need to be addressed.

Further research and development

In mid 1996 a further series of trials are planned to be conducted with a corporate partner. These trials will evaluate the Beta version of the Advance system through the delivery of training to people in the workplace. The aim of these trials is to test the applications effectiveness of Advance products and to evaluate other JITT models that are possible through the Advance system. Considerable more research is required to clearly demonstrate the educational outcomes of wide scale use of online multimedia training delivery systems.

References

Advance Project Report, No. 1 (1995). MTE prototype classroom stage 1, February 1995, evaluation report. Computer Power Education.

Advance Project Report, No. 2 (1995). Applications effectiveness test report. Computer Power Education.

Advance Project Report, No. 3 (1995). Multimedia course developers guide. Computer Power Education.

Advance Project Report, No. 4 (1995). Advance prototype evaluation report. Computer Power Education.

Authors: Dr Michael Ross
Project Manager - Education
Advance Project, Computer Power Education

John Merakovsky
Senior Consultant Business Education
Advance R&D Project, Computer Power Education

493 St Kilda Rd, Melbourne 3004
Ph (03) 9243 2320 Fax (03) 9820 2010
Email: michaelr@cpe.com.au

Please cite as: (1996). Your mentor on the superhighway: Just in time multimedia training for corporate and government. In C. McBeath and R. Atkinson (Eds), Proceedings of the Third International Interactive Multimedia Symposium, 371-380. Perth, Western Australia, 21-25 January. Promaco Conventions. http://www.aset.org.au/confs/iims/1996/ry/ross.html


[ IIMS 96 contents ] [ IIMS Main ] [ ASET home ]
This URL: http://www.aset.org.au/confs/iims/1996/ry/ross.html
© 1996 Promaco Conventions. Reproduced by permission. Last revision: 15 Jan 2004. Editor: Roger Atkinson
Previous URL 6 Jan 2001 to 30 Sep 2002: http://cleo.murdoch.edu.au/gen/aset/confs/iims/96/ry/ross.html