IIMS 96 contents
[ IIMS 96 contents ]

Multimedia performance support systems for industry

Allan Briggs
Central Oueensland Institute of TAFE


Theoretical underpinnings of electronic performance support

To begin with I would like to highlight some of the theoretical underpinnings of Electronic Performance Support Systems. Gloria Gery started the ball rolling in 1991 with her seminal work titled, 'Electronic Performance Support Systems'. Her vision of 'just in time' training is becoming more of a reality as time progresses.

Gery (1991) talks about the historical movement of training away from reality and into a virtual environment, where 'training (but not job learning) became an event conducted in a classroom with images on overhead transparencies representing reality.' This was certainly the case in 1991 when Gery wrote her book. I can myself recall teaching compressed air brake air drier systems entirely theoretically because we did not have an example of a system to show the students. The topic was in the syllabus and it had to be covered. The students could not see the real components nor could they practice any skills. They had to be satisfied with the reality of coloured lines on a sheet of acetate.

Things have changed in recent times with the advent of competency based training and the requirement that students perform tasks to a competent standard on real equipment. This has certainly moved the pendulum back towards training within the reality of competencies which are performed on the job. However, these competencies are still performed within the artificial environment of a college and are generic in nature, not specific. Students are required to be able to apply these skills in the workplace. They have to apply the skills of using the spreadsheet they learned at college to that of using the spreadsheet preferred by the employer. They have to apply the skills of repairing the Ford engine they worked on at college to that of repairing the Holden engine the employer wants them to work on. There has been nothing historically wrong with this, after all it is the basis of most technical and further education. However, in an increasingly complex and technically demanding working environment is it still appropriate?

Gery (1991) comments 'many jobs today have become more complex, involve more variables, and require more analysis, synthesis, and interpretation in increasingly diverse contexts.' In this situation does the average course provide the average student with the skills required for the workplace? If they are competent in the generic skills provided by a course of study will they be competent in the more specific and complex tasks demanded in the workplace? The answer may well be that while they are not immediately competent it is hoped they soon will be by virtue of the transfer of knowledge and skills.

The next question that arises is to ask if mere competency is sufficient? Gery thinks not. She suggests that mastery is the goal that quality and benchmarking programs aim for. That the marketplace is demanding more than just the competence to take off and land. It wants the mastery that is demonstrated by aerial acrobatics. This scouring of the marketplace for high flying ability is illustrated by one of my clients who has initiated a preferred provider program for training providers. Those who have demonstrated the ability to meet specific criteria are signed up and get first choice of supplying training to meet company requirements. The merely competent training provider is left to fly a kite.

The solution according to Gery (1991) is not to provide more training or even smarter training in the form of Computer Aided Learning. The solution is to provide 'on demand access to all of' the resources individuals need to solve a problem, perform a task or - some day - to do an entire job.' This is what Gery refers to as 'living in the performance zone'.

Gery (1995) more recently has put forward the notion of Performance Centred Design which represents the evolution of performance support from an add on extra to total integration with the design of software. Some of the examples which are given include Microsoft Publisher and Quickbooks by Intuit. Her recent work has also included the development of nineteen attributes of an EPSS. This includes such things as 'Provide contextual feedback', No 11 and 'Automate tasks', No 15. These provide a very useful framework for a design methodology.

Most of these considerations focus on the system itself, how to provide information, structures and mechanisms by which information can be organised, design of the GUI, use of databases, etc. But what about the human attributes of performance which these systems are supposed to enhance? A closer look at these attributes may help us to design a better system. Gary Dickelman (1995) exhorts us to refer to the work of Donald Norman (1993) who has looked at these attributes and has some very interesting things to say. Although Norman does not write specifically about EPSS his work on 'human attributes in the age of the machine', could be useful in the design of an EPSS.

Norman's view is that the computer screen is a virtual environment made up of artefacts placed there by the designer. Performance is a function of how well these artefacts represent reality. Good design would focus on three principles.

  1. The representation must match the task!
  2. The artefact must match the person!
  3. The passive constraints that are imposed by artefacts must focus (abstract) the critical features of the "world"!
In short the essence of good design as Norman (1993) puts it, 'is to start with the needs of human users of the system, not with the requirements of technology'. This will avoid the problem which concerns Dickelman (1995) of having lots of hypertext, radio buttons, popup notes, video clips, help files, wizards and genies which in effect do not support performance but rather confuse the user.

Personally, I am very excited about the potential of a system which can enable us to live in the 'performance zone'. Professionally, I am numbed by the enormity of the task. However, I am not daunted by it. The challenge of an EPSS is not just to provide information but to fit it tightly around the context of the performance problem.

The best way to test a theory is to put it into practice. Recently I had the opportunity to do this for a client in the 'real' world of the mining industry who had a 'real' performance problem. I will proceed by describing how I applied some of the principles already discussed to the design of an EPSS.

Producing an electronic performance support system

A performance problem in relation to the repair of faults on the hydraulic system of a Hitachi face shovel was identified by Tarong Coal, a coal mining company which is part of the CRA group. A traditional training solution was proposed but due to the complexity of the hydraulic systems it was perceived that ongoing support after the training event was crucial to the maintenance of performance. This support had to be cost effective and meet the needs of the tradesperson on demand and at the time of need. The strategy of using an EPSS was considered and chosen as the best means of providing support.

There were several reasons for this decision.

One major constraint was the budget figure of less than $20,000. This was not generous and would have an impact on the design. However, it was felt that a tight budget would concentrate the design around essential features and keep the EPSS focussed on meeting real needs.

Once a decision was taken to use an EPSS the following goal was established.

To provide graphic, textual and animated information of the operation and testing of the hydraulic circuits on a Hitachi EX3500 that will assist a tradesperson in performing testing and adjustment procedures to manufacturers specifications.
It was also decided to centre the design criteria around the nineteen attributes of an EPSS as described by Gery (1995). While it was realised that it may not be possible to incorporate all of these attributes it was decided that they represented best practice and worthy of emulation. These attributes are as follows: In addition it was decided not to lose sight of the fact that the system would have to meet the needs of the users, thus embellishment without purpose would be counterproductive.

Even though the design criteria provides a focus for development it does not provide a methodology. The question was wether to use an existing Computer Aided Learning (CAL) methodology as developed by the likes of Alessi & Trollip (1991) or maybe a totally different methodology was appropriate. The elements of CAL are similar to the elements of an EPSS, graphics, hypertext, video, animation and sound. From this point of view a traditional methodology of analysis, design, development and evaluation would be appropriate. Gery (1991) makes the comment that 'EPSS is a concept not a technology. It takes advantage of whatever technologies are appropriate.... It is therefore difficult to define a detailed development methodology for EPSS'. Gery does however, provide some guidelines for establishing a methodology.

From my point of view there are important differences between CAL and EPSS which have to be considered.

  1. The interface has to provide instant access to whatever information is required by the user. If that information is on screen nine of module ten the user should be able to access that screen with a few simple mouse clicks.

  2. Assessment will be related directly to the task and will often be procedural in nature. The user requires performance support practice not tests of arbitrary knowledge. From this point of view a dry run of a procedure with feedback on performance is an example of what assessment should be about.

  3. Strong support for the interpretation for real problems rather than hypothetical ones would be critical. This could take the form of what to do when a particular problem arises, what tests to carry out, what equipment is needed, branching to take account of all known variables, advice on causes and remedies.

  4. Embedded information which is only accessed when required. The difference between training and performance support is that training takes the trainee through all of the steps in acquiring knowledge and skills, performance support provides access to the knowledge base and application of skills, but only if required. The user decides what information is needed and when it is needed.
I will proceed by describing how the finished product was produced and draw some conclusions about the effectiveness of the EPSS in meeting the established goal and design criteria. I will also examine how the development plan, which was based around the model developed by Alessi & Trollip (1991), and which took note of the guidelines provided by Gery (1991), worked in practice. I will discuss each step of the development process and explore those areas where the plan was sound as well as those where some deviations had to be made, and how the attributes described by Gery (1995) were addressed.

Determine needs and goals

A needs analysis had been conducted which determined eleven specific tasks to be performed by the tradespersons. The importance of a clear identification of such tasks cannot be overemphasised since in the words of Gery (1991) 'the focus of the entire analysis and planning phase must be the needs of the performer or learner in relation to the tasks, content or processes.'

The established goal is set out above and was able to be met quite successfully. From this point of view the goal was neither ambitious or cautious, it fitted the needs and was achieved within budget.

As far as user goals are concerned it was decided that these referred to troubleshooting. How the user determines these goals is discussed in more detail below.

Collect resources

Physical resources

Physical resources for EPSS are directly related to the technology selected for the EPSS. For example video requires a video camera, a troubleshooting adviser requires an expert system. The selection of a suitable technology for each aspect of the EPSS will be a crucial factor in the usefulness of the information to the user. For example if the user needs to be able to identify a component, a description of where that component is may not be adequate. A digitised photograph of the component together with the description will better enable the user to find the component.

The attributes of each technology and the information it can contain need to be well understood. For example why use video of an inanimate object when a still image can provide the same information.

For our project the selection of suitable technology was important from a budgetary point of view and careful analysis of technology attributes enabled a focus on user needs to be maintained. There is also the consideration that the selection of a particular technology will require necessary human resources to be able to use that technology. There is no point in selecting virtual reality if the skills required for the development of virtual reality are not available within the team or the budget cannot stretch to employing a consultant.

For our project we stayed within the boundaries of the physical and human resources that were available.

Human resources

Whilst a team approach was used the team was not large. A project manager kept the project on track and on budget. The instructional designer and programmer were one and the same person. A part time graphic artist was used to produce all of the line drawings of hydraulic circuits.

Learn the content

With the exception of the graphic artist the team members had good subject matter knowledge which was a major factor in being able to complete the project within budget and on time. Obtaining subject matter experts who have the time available to devote to a project is always a difficult problem. In most cases the client will nominate an employee with appropriate skills but will not be able to release that person from normal duties. Being able to provide subject matter expertise within the development team is a good selling point and means that development can be continuous rather than sporadic.

Whilst learning the content was not particularly difficult it was found that obtaining the necessary detail was more difficult than anticipated. With an EPSS getting the details right was most important. In many cases the service literature did not supply such detail and it was necessary to actually perform many of the tasks to ensure that procedures were correct.

Generate ideas

Brainstorming was a very useful exercise which was resorted to more than once. As well as generating the initial format of the EPSS it was found that brainstorming was able to solve several problems encountered during development.

Ideas which came out for the original design included

  1. break the EPSS into manageable parts which could be developed separately. These parts would relate directly to the eleven key tasks and also allow the user to select that part relevant to their needs. Attribute 7, 12 and 13.

  2. develop a template which could be used for each part to maintain a cohesive design structure. Attribute 19.

  3. use embedded pop-up text to keep screens uncluttered and also to maintain the EPSS principle of information as required. Attribute 5.

  4. use hot spots on graphics to provide relevant information about parts of components ie. to identify inlet and outlet ports. Attribute 19.

  5. provide a 'content selection' combo box which would allow the user to select any part of the content from a list. This would enhance the speed of access to information. Attribute 3.

  6. provide a 'glossary' combo box which would contain a technical glossary of hydraulic terms and be available anywhere in the program. Attribute 16.

  7. use of a menu screen which would allow users to select any one of the eleven key tasks. Attribute 3.

  8. provide an introduction section so that new users could become familiar with how to use the EPSS. Attribute 1.

Design instruction

The design of instruction was centred around the tasks which were identified as being necessary to support the performance of maintenance staff in carrying out repairs. Concentrating design around these tasks kept the focus of the EPSS on user needs. Eleven key tasks were identified and I will describe how the design of the EPSS was structured to provide instruction in each one.

Identify all hydraulic components on the machine
The use of captured video frames saved as bitmaps provided the basis of the information for this task. Being able to identify components meant being able to see them so graphic information was essential. A bitmap identifying the component is provided together with a short description of what the component is, where it can be found, its function and some information on its operation. Attributes 5 and 6.

A 'Content Selection' combo box in the bottom left corner of the screen provides a pop-up list from which any component can be selected. Attribute 8.

Carry out scheduled oil sampling and routine servicing
This mainly required a table of information with servicing intervals, filter types and oil types. Information on how to take an oil sample was provided in other screens. Attribute 5.

Describe the function and relation of all hydraulic components in each circuit from symbolic diagrams
Because of the complexity of the hydraulic circuits it was decided to use a composite block diagram showing the relationship of all components and thereafter to show individual circuits separately. This worked well in reducing the confusion factor associated with complex diagrams and also allowed hot spots to be placed on the diagrams which are associated with popup text. This increased the amount of information available to the user but was embedded in the screen and only visible if selected. Attribute 5.

Figure 1 shows a screen from the 'Circuits' topic. This is the composite diagram. As the user moves the cursor over the screen it will change to a hand when it encounters a hot spot. At the same time the name of the component will be shown on the status bar at the bottom of the screen. If the user clicks on the hot spot a pop-up text box will appear providing a description of the component.

Figure 1 shows a standard screen layout for the EPSS. It contains a 'Contents Selection' combo box which provides hyperlinks to other parts of the program. It also contains a 'Description of Terms' combo box which provides a glossary. First page, last page and page forward or back buttons are provided within each topic and the page number is also indicated. A 'Main Menu' button is provided on every screen. Attributes 5, 13, 16 and 19.

Figure 1

Figure 1: Standard screen layout

Describe the operation of undercarriage circuits
Describe the operation of front attachment circuits
Describe the operation of swing circuits
Describe the operation of pilot circuits
Describe the operation of travel circuits

These five tasks all required a similar treatment. Each required a detailed explanation of circuit operation. Tasks a) and c) provided an introductory overview to the inter-relatedness of components and circuits whereas these five tasks, d) to h), would have to provide substantive detail. For this purpose hydraulic symbols would be necessary and animation of those symbols would clearly show circuit operation. Attributes 5 and 8.

In order to connect the operational information to the test information hyperlinks are provided to those parts of the EPSS where these tests are described. Attribute 5, 12 and 18.

Describe the procedure for carrying out individual hydraulic circuit pressure tests.
There are many pressure tests which can be carried out on the hydraulic system to determine if pressure settings are normal. The intention here was to provide the user with a list of tooling required for the test, specification data, a description of the procedure and a diagram showing where the pressure gauge had to be connected. Attribute 3, 5, 7, 18.

Once again the 'Contents Selection' combo box can be used to directly select information on any pressure test which can be carried out on the machine. Attribute 12.

Describe the procedures for carrying out implement performance tests.
This task is similar to i) and the same approach was used. For example the No-Load Travel Speed screen from the 'Performance Standards' topic shows a bitmap of how the machine should be positioned for the test and a button under the bitmap allows the specifications to be displayed. Attribute 12.

Accurately diagnose faults using a logical troubleshooting procedure
This was a crucial part of the EPSS and deciding how to structure the information required careful analysis of what the user would need. After studying troubleshooting principles put forward by Kepner Tregoe (1985) it was decided that the user need could be characterised by the question, 'Given these fault conditions what should I do'? From this a troubleshooting adviser was designed which will allow the user to select fault conditions and then enter a phase of testing where certain tests are advised, the results of which can be expressed as yes or no. This will allow a successive phase of branching until all tests have been exhausted and the fault identified.

For example if the user selects, 'The machine will not travel', as the fault, one of the tests would be to test the travel speed control pressure. Instructions on how to carry out the test would be offered, if required, an t e question, 'Is the travel speed control pressure within specification', would be asked. The yes or no answer will then branch the user to the next part of the test sequence. This process would continue until the adviser is able to suggest a cause of the fault. Attributes 2, 3, 5, 6, 7, 9, 12, 15, 16 and 18.

This part of the EPSS is still under development and has proven to be the most difficult to construct. However, early trials have suggested that it does indeed meet the need for which it was developed.

Underlying design

The EPSS has been designed like the layers of an onion. As you peel off each layer you get closer to the core of the matter. For example if there is a need to troubleshoot a problem with the travel circuits, a look at the topic 'Circuit Diagram' would explain the relationship of circuit components to other circuits, the topic 'Components' would show you the travel circuit components and explain where they were on the machine, the topic 'Travel' would provide a detailed explanation of circuit operation, the topic 'Performance Standards' would enable you to check if the travel circuits were operating satisfactorily, the topic 'Pressure Tests' would allow you to test the travel circuit and the topic Troubleshooting' would provide advice on troubleshooting procedures and tests if needed. Hyperlinks provide a connection between each topic and allow the user to use a free form method of navigation.

Topics can be selected as the user requires since the user may already possess some knowledge of system operation and testing procedures. The user may prefer to investigate the problem using their own logic and prior learning instead of relying on the troubleshooting adviser. In this sense the user has total control over the selection and use of information.

This aspect is designed to reflect the needs of the user in going about the task of finding a fault. The interface is simple and direct, it is also hoped that it is intuitive.

Flowchart the EPSS

The flowchart which was developed in the design stage was altered many times as the need for various hyperlinks became apparent. In this sense the EPSS is an evolutionary beast which was born of sound design and is still in its infancy.

Storyboard

The storyboards were very effective and certainly focus ideas during the design stage. However, it was found that many changes were made on the fly during development. These changes were necessary when the planned screen layout did not work or some rearrangement of screen layout was necessary. Most of the problems were the result of an entrenched CAL mentality where all information is presented to the user. In an EPSS the user selects what information is needed and re-design centred on embedding information rather than overtly displaying it.

Programming

Assymetrix Multimedia Toolbook was found to be more than adequate to the task. It is a powerful authoring tool which can do just about anything a design can dream up. The limitation is of course not in the software, but in the skills of the programmer. Advice offered by the forum of the Toolbook User Group at a listserv address serviced by Arizona State University proved to be invaluable. Several intractable problems were found to have simple answers when experts were consulted. The power of this forum has both amazed me and provided access to developers who have often encountered similar problems and are more than prepared to share their solutions. In many respects this forum has all of the hallmarks of an EPSS where solutions to problems are 'just in time'.

Produce Support Materials

The EPSS is in fact a support for the course itself and as such represents the support materials in this case. However, the main course materials were produced in conjunction with the EPSS and references from one to the other were made. The EPSS was used during training as a normal piece of CAL and served a useful educational purpose as well as introducing users to the EPSS and how it was to be utilised subsequent to training.

Evaluation

As well as the usual developmental evaluation process the usefulness of the EPSS in supporting performance has also to be evaluated. This can be best done by measuring performance before and after the implementation of the EPSS. In our case maintenance statistics in the time taken to repair hydraulic faults will provide the data. Analysis of this data will provide evidence of wether we have been able, in the words of Gery (1991), to provide 'user support, rather than developer amusement and challenge'.

Conclusion

In general I am very pleased with the finished product. The goal has been met and the design criteria provided a solid and practical foundation for development. In fact the robustness of the design was quite a surprise since I expected there to be many more changes than was experienced. There is certainly room for improvement and indeed many of the improvements are planned for an update in 1996.

Feedback from users has indicated that the EPSS has more than fulfilled the need for which it was designed. It has become the first point of reference for supervisors and tradespersons. It has also been installed on a notebook computer which is taken out to the machine whenever a hydraulic problem is encountered. This is very satisfying and has given me some confidence in looking at ways in which the EPSS can be further improved.

I think that the original decision to use an EPSS as an enhancement of the training program has been justified. It has been used throughout the training program as well as subsequent to it and has become as much a tool as a pressure gauge in the diagnosis and repair of hydraulic system problems. It is the complete acceptance of the EPSS as an integral part of the toolkit which is most pleasing.

When I reflect upon the genesis of the EPSS I am aware that it was not only born of the stated goal at the beginning of this paper but also of a goal given by Gery (1991) which states, The goal of an electronic performance support system is to provide whatever is necessary to generate performance and learning at the moment of need.' My efforts in this direction have produced something which is useable and is being used to support performance. However, I am conscious that there is some distance to go before we can say that we have provided 'whatever is necessary' at the 'moment of need'.

References

Alessi, S. & Trollip, S. (1991). Computer-based Instruction: Methods and Development. Englewood Cliffs, NJ: Prentice-Hall.

Dickelman, J. D. (1995). Things that help us perform: Commentary on ideas from Donald A. Norman. Performance Improvement Quarterly, 8(1), Special edition published on the Internet by Learning Systems Institute, Florida State University.

Gery, G. J. (1991). Electronic Performance Support Systems. Boston: Weingarten Publications.

Gery, G. J. (1995). Attributes and behaviours of performance-centred systems. Performance Improvement Quarterly, 8(1), Special edition published on the Internet by Learning Systems Institute, Florida State University.

Kepner-Tregoe. (1978). Analytic Trouble Shooting. New Jersey: Princeton Research Press.

Norman. D. A. (1993). Things that make us smart - Defending human attributes in the age of the machine. Reading, MA: Addison-Wesley.

Author: Allan Briggs
Central Queensland Institute of TAFE
Rockhampton College
Locked Mail Bag 8065
Rockhampton Queensland 4700
Telephone: (079) 314 615 Fax: (079) 223 756
Email: abriggs@deakin.edu.au

Please cite as: Briggs, A. (1996). Multimedia performance support systems for industry. In C. McBeath and R. Atkinson (Eds), Proceedings of the Third International Interactive Multimedia Symposium, 73-79. Perth, Western Australia, 21-25 January. Promaco Conventions. http://www.aset.org.au/confs/iims/1996/ad/briggs.html


[ IIMS 96 contents ] [ IIMS Main ] [ ASET home ]
This URL: http://www.aset.org.au/confs/iims/1996/ad/briggs.html
© 1996 Promaco Conventions. Reproduced by permission. Last revision: 14 Jan 2004. Editor: Roger Atkinson
Previous URL 26 Nov 2000 to 30 Sep 2002: http://cleo.murdoch.edu.au/gen/aset/confs/iims/96/ad/briggs.html