[ EdTech'94 Contents ]
[ EdTech Confs ]
Towards successful computer based learning - the complete process
Adrianne Dymalla, Bernadette Kent and Ann Russell
University of Queensland
A computer based learning (CBL) project can be considered successful if it meets its educational objectives, fulfils a need within an organisation, and is cost and time effective in development. It is the process of planning and developing successful CBL which we endeavour to address in this paper
Of primary importance is the issue of planning. The adage "If you fail to plan, you plan to fail" could not be truer than it is of Computer Based Learning (CBL) development. As Gery (1987:94) asserts: "No other medium is so unforgiving of sloppy planning". Yet, possibly due to the ease with which one can quickly develop something that works using the current authoring languages, many CBL developers begin developing with little or no project planning. Our paper aims to describe a model of a CBL project life cycle from the initial needs analysis phase through to the learner delivery phase. The life cycle of a CBL project is best displayed diagrammatically (Figure 1)
Figure 1: Life cycle of a CBL project
For the purpose of this paper. we are assuming that the client is in some way external to the development group, either as a separate commercial organisation or a separate department within an organisation or educational institution. However, the premises made here can be adapted to other CBL development situations.
Phase 1: Needs analysis
Initial consultations
From the first time the client meets with a representative from the development team, important decisions about the scope of the project will begin to be made. For this reason, it is crucial that all parties who need to have an input into the decision process are involved from the outset.
Who should be there?
- the client and content expert,
- the manager of the development team, and
- a courseware developer/programmer.
Why these people?
- The client is trying to meet a need within his/her organisation which is also providing the financial backing to produce the program.
- The content expert needs to have a good understanding of the life cycle of the project and be aware of their role within the development team.
- The manager of the development team needs to have a good overview of the project. Later, in consultation with other members of the development team, the manager must set out a project schedule and determine costs.
- The courseware developer/programmer contributes to the meetings by answering or asking any questions about the project which are relevant to software, hardware, and instructional design decisions.
Given the specialised nature of CBL development, it is essential that representatives of the development team include someone who has a working knowledge of hardware and software capabilities as well as instructional design. Failure to do this may result in promises being made to clients which cannot technically be delivered, commitments being made to time frames which are not realistic, or instructional design decisions being made which do not consider the CBL medium.
What does the development team need to ask the client/content expert?
- what are the end user/learner characteristics?
- what is the need/purpose/educational objective of the program?
- what form do they envisage the program will take, eg. drill and practice, tutorial, simulation, a combination of these?
- how is the courseware to be tested and evaluated?
What does the development team need to communicate to the client/content expert?
- a basic explanation of CBL and its capabilities, may be achieved by demonstrating existing CBL programs in a variety of forms,
- CBL terminology, eg. module, menu, interactive
- the process of the project life cycle.
Both the client and developer should discuss and agree upon the various stages in the life cycle of the project, particularly what involvement level will be required of the client/content expert. This will probably include an outline of the tasks of the project, who will be responsible for them and points at which the client/content expert can request changes. Milestones should also be established where the client/content experts' signature indicates their satisfaction that a phase is complete.
Defining the purpose
It is essential to identify the objective at the outset. The very first question that should be asked is "What do you want this CBL to achieve?" The answer may vary from very specific learning objectives to providing a student assessment tool. Objectives should be stated in terms of learner or user outcomes, rather than in terms of technology. That is, statements like "I want to develop a multimedia package with video" should be avoided. This objective is related to technology, not education or fulfilling an organisational need. When the objective is technology based the client runs the risk of spending a lot of money and time developing a very impressive white elephant. If an objective is stated in terms of a need or educational objective, it can then be decided if multimedia with video is an appropriate tool to meet this objective.
Phase 2: Project specifications document
Once the objectives are clearly determined, the project can be defined in the form of a specifications document. The specifications document will result from one or more meetings with the client and from a number of drafts which will be reviewed and rewritten until all parties agree to its contents. The purpose of this document is to establish guidelines, standards and requirements so that once development has begun, it can be continually measured against these specifications. The danger of not producing such a document and either not discussing the issues, or relying on verbal agreement, is that there are then no tangible guidelines for team members to use. This means it is possible for the "goal posts" to be shifted by either party (ie. the developer or the client). potentially adding considerable time and cost to the project. A guideline of what should appear in the specifications document follows.
Project specifications
1. Statement of purpose
Without a clear statement of purpose it is easy for a project to change direction midstream. This can occur when the client feels that the product is "not what they had in mind". It is therefore essential to document at the outset what they do have in mind, so that if this objection is raised at a later stage, it can be determined whether the project is following what was originally agreed upon or whether they have changed their mind.
The educational objective or organisational need which should have been determined in the needs analysis phase should be clearly stated here.
2. End user requirements
As stated in the needs analysis, the end user characteristics should be identified. These characteristics will dictate not only what happens in terms of the content and style of the program but also its nature in terms of delivery requirements.
For this reason some of the issues directly related to the delivery phase must be addressed now. These include decisions on:
| Hardware: |
PC/Mac/Unix etc
Screen resolution (VGA etc.)
Multimedia device drivers (sound card, video card, CD-ROM etc.)
Network requirements
Memory requirements |
| Software: | Compatibility with existing software - Windows/DOS etc.
Licensing agreements |
| Records Mgt: | Necessary if for assessment purposes. |
| Production: | Cost of copying disks, CDs etc.
Packaging and labelling
Installation |
| Documentation: | User manuals
Technical manuals
Supplementary educational material |
An important delivery consideration is the "lowest common denominator" in terms of hardware the user has available to them. For example, high resolution graphics could not be displayed on a monochrome monitor. On the other hand, going for the lowest denominator may compromise the effectiveness of the CBL.
Industry hardware standards should also be considered if the CBL program is to be sold or distributed widely. For example, if sound or video is used, then compatibility with a common brand of sound or video card is desirable.
3. Scope of the project
Once the audience and purpose of the CBL project is established, the nature of the project can be broadly defined. For example, if it has been established that a package is needed to teach workplace health and safety policies to people with poor literacy skills, it may be important to develop a package that makes extensive use of sound and graphics.
More specific requirements for the project can also be defined at this stage. These may include: content topics, flow and structure, style, menus, branching for feedback, paging forward and backward, types of graphics, results recording and reporting. These factors should all address the educational objectives or need of the CBL. For instance, record keeping facilities are often incorporated into a program without any real purpose or educational value.
4. Cost and time scheduling
The most successful CBL will result from defining the educational objective and audience first, identifying an approach that will meet that need, then pruning or expanding on that approach according to budget. There seems to be a mind set which says the less money the client has, the less interactive, exciting and innovative the product can be. While to a degree this is true (there is a big difference in cost between "full blown" multimedia and a simple Computer Managed Learning package), a better product will result if the educational objective is the focal point of the project. The result may be ten minutes of interactive, interesting and educationally useful CBL rather than an hour of electronic page turning which does not help students learn.
In addition to cost considerations, the scheduling of a CBL project is often very difficult. This is partly because many of the development methodologies are new and the technical nature of the process can mean "technical delays". However, it is our experience that most delays are due to poor planning or because the nature of the project changes significantly during development. If the boundaries of the project are clearly defined at the outset, roles and responsibilities determined and milestones and deadlines identified, the time schedule can be much more tightly controlled.
With the information gathered during the initial consultations the project manager can now establish a list of tasks which will be necessary to complete the project. These include sequencing and estimating the time for each task to be completed. This will lead to a time schedule which will quickly show how soon the project can be realistically completed. If it conflicts with the clients' desired completion date, negotiations can be made about the nature of the work to be carried out or extra money or resources being assigned to enable the project to be completed in time. It is also very important for the project manager to take into account the learning curve associated with ever changing technologies when determining the schedule.
5. Role definition
Successful CBL projects are invariably the result of good team work. The fact that team members may perform several functions on a project does not alter the fact that the processes involved are quite distinct and must be clearly defined. From our own experience, problems have occurred when responsibilities were not clearly defined. On the one hand, certain tasks may be neglected if there is confusion over where responsibility lies. On the other hand, a certain amount of "reinventing the wheel" and bruising of professional pride may occur if team members overlap on certain tasks. This is not to say that ideas and experience should not be shared, but the responsibility tor each task must be clear.
Obviously the size of the team depends on the nature of the project but we have found that an effective mix of personnel and skills enables responsibilities to be assigned as:
| Role | Responsibility |
| project manager | oversee project, set deadlines, etc. |
| instructional designer | guide content expert with script/storyboards and educational objectives |
| content expert | provide script/storyboards and educational objectives in conjunction with instructional designer |
| programmer | coding of storyboards and utilities (eg. student records) |
| graphic artist | design of screens/ graphics/ supplementary materials |
| technical writer | produce documentation |
| technical specialists | as required (eg. video production, network setup) |
Everyone involved in the project should be made aware of the time commitments required of them and the impact that failure to meet preliminary deadlines will have on the project deadline as a whole.
The role of the content expert, in particular, requires an ongoing commitment throughout the life of the project. The content experts role includes:
- writing the educational objectives,
- deciding what information should be presented,
- scripting the storyboard and determining the structure of the program in conjunction with the instructional designer,
- being available to answer queries regarding the accuracy of the content, branching, graphics, and animation,
- review and sign off of each phase of the project.
6. Sign off
Once both parties have agreed to the content of the Project Specifications Document, it should be signed by both the project manager and the content expert or client.
Phase 3: Design and storyboard
The storyboard allows the content expert and development team to verify the content, language and sequencing before the programming begins. It also allows additional resources to be identified, such as specific graphics, sound and video sources. Well written storyboards can significantly reduce the need to make extensive editing changes during the development phase of the project. For this reason it is important that the instructional designer show the content expert examples of storyboards and discuss storyboard techniques.
Storyboards are a representation of each screen. They may be prepared on a paper based template, similar to the template displayed in Figure 2, or as a word processed script. They provide space for text and graphic descriptions, programming directions and directions for learner interactions. Storyboards ensure the content and design is consistently displayed throughout the CBL program.
Navigation keys
Screen Details:
Screen number:
Branching information:
Special keys: eg. F8 = Glossary
Graphic Information:
Video:
Sound:
Figure 2: Storyboard template
Similar format rules apply to a storyboard as to an overhead transparency. Text is broken down into point form wherever possible and large paragraphs of text should be avoided. Other suggestions are to write short. concise. complete sentences, be direct, use a conversational style, include precise directions, use correct spelling and grammar. try to limit the amount of new information to one or two ideas per screen, and avoid abbreviations unless they are well known to the user. The information displayed on each screen can be divided into the following screen types.
Introductory screens can include:
- a title screen.
- a credit screen,
- a statement of the general purpose and/or educational objective of the program, an overview of the how the program works, and
- menus.
Information screens can be displayed in a number of ways depending on the content and desired effect. For instance:
- facts can be presented in a matrix
- procedures can be displayed by a series of steps, providing practice and feedback
- rules which are governed by cause and effect relationships can be displayed by providing examples of correct and incorrect application of the rules, practice and feedback
- concepts can be conveyed by definition, examples, negative examples and analogies.
Interactive screens ask the learner for a response and provide immediate feedback whether correct or incorrect. When incorrect the learner should be given some remedial feedback or be directed to further information. Simulation screens allow the learner to practice a procedure in a format as similar as possible to a 'real world' procedure. Summary screens conclude the lesson and may direct the learner to further modules or information. Once story boarding is completed a prototype of the proposed program can be developed. The prototype can be made up of several screens including a sample graphic, animation and interactive screen. It allows the content expert and client to get a glimpse or feel for the end product. Any misunderstandings, suggestions, or comments can then be finalised and signed off before development begins.
Phase 4: Development
The development phase may consist of several activities, many of which may be happening in parallel. These may include:
- authoring - transferring the storyboards to the screen;
- programming - of record keeping utilities, simulations, complex interfaces etc.
- graphics - title pages, images, animations etc.
There are three factors governing the success of the development phase:
- preparation - the project must have been well planned before development begins;
- role definition - each task should be clearly defined and allocated; and
- communication - team members must be able to communicate effectively with each other so that all the tasks integrate successfully.
Once the development phase is started then in theory there should be no further changes. This is rarely the case. As the program develops the content expert/client or even the development team, may decide that adjustments need to be made to clarify information. The project schedule should be flexible enough to allow for these inevitable minor adjustments. Changes should be dealt with via a Request for Change Form, so that changes can be expediently documented, located and signed oft. A suggested design for the Request for Change Form is displayed as Figure 3.
Figure 3: Change Request Form
Probably with CBL more than any other discipline it is possible for one person to assume many roles. Indeed there are authoring products on the market advocating that a single person can sit down at a computer and produce CBL programs "on the fly". This may be true for multi-talented individuals with lots of time on their hands but in reality, each of the skills required is quite specialised and time consuming. It is typically the right mix and management of these specialised skills that produces successful CBL.
Phase 5: Testing/evaluation
It is expected that the greatest number of recycles in the project life cycle will occur between the development and testing phases. The testing phase consists of two types of testing. Firstly, technical testing consists of checking that the program functionality works. For example, the program quits when the quit button is clicked, goes to the menu when its supposed to, help screens are in place and student records are saved correctly. The second type of testing is the content testing. This means that the correct information is presented to the learners in an unambiguous manner. If the story boarding phase was given sufficient attention, there should not be too many problems here. The best way to achieve this testing is to trial it on some sample learners, other content experts and developers.
A Request for Change Form will formalise any request for amendments and must be signed off. It must be made clear that changes at this stage must only be made to correct mistakes or to enhance the educational outcomes. Changes should not be made on a whim. If this phase is not controlled, it can easily get out of hand as more and more people offer suggestions for improvement resulting in a "blowout" of cost and time. The bottom line is: if it is technically correct and educationally sound then leave it alone.
Evaluation of the project after it has been in use tor some time is also desirable. For example. to see if the CBL has improved overall student results within a course. This type of evaluation is usually outside the realm of the of project life cycle but is mentioned here for completeness. The end user is the final judge of the product so ongoing evaluation and amendments are crucial to the success of the product.
Phase 6: Delivery and closure
As the name implies, delivery means the project can be handed over to the client and signed off. At this point there should not be a cycle back to previous phases of the project life cycle. Basically, there should not be any surprises at the delivery phase.
If delivery was not considered in the initial planning phase the entire project may be in jeopardy. For example, suppose a CBL program was to be used by students in a university library and it was not able to be shared over a network then only one student at a time could use the CBL. To live with the situation could be frustrating, to change it could prove costly and time consuming.
If the project has been properly planned and managed, then the delivery phase should go relatively smoothly. One comforting thought is that as time goes by, more and more products are becoming "standardised". This may mean that compatibility at the delivery phase will become less of a headache in the near future. However, there will never be a substitute for adequate planning.
On completion of the project, the development team along with the project manager should have a meeting to discuss the project. This is a good time to review the entire life cycle of the project and revise procedures where necessary. Closure of the project is necessary to ensure that team members can progress to the next project with a constructive positive attitude.
Conclusion
The degree of planning outlined here is not always looked on favourably by the client and even, at times, by higher management. People can be perplexed about spending so much time up front and on paper when developing computer based materials. However, again, as Gery says:
You either pay up front by following the precise and demanding development steps, or you pay later with less than adequate programs, several passes through the revision or redevelopment process, or poor learning outcomes with resultant poor learner performance on the job (1987:95).
Recognition that there is a great deal more to the process of developing CBL than dealing with the technology is the first step towards success. Overcoming the desire to start developing without planning is another. The complex range of tasks, personnel and resources which comprise a CBL package, requires careful planning and integration. Good planning and project management are more likely to result in successful CBL; meeting the need of the client in time and on budget.
References
Gery, G. (1987). Making CBT Happen: Prescriptions for successful implementation of computer based training in your organization. Boston: Weingarten.
Meredith, J. R. and Mantel Jr, S. J. (1985). Project Management, A Managerial Approach. New York: John Wiley & Sons.
Richards, J. (1990). Program Design. Workshop delivered at the University of Queensland. St. Lucia, June 20.
| Authors: Adrianne Dymalla, Instructional Course ware Designer, Tertiary Education Institute - CAL, University of Queensland, 4072, Brisbane, Australia. Phone: (07) 365 2322; Fax: (07) 365 1799; Email: a.dymalla@mailbox.uq.oz.au
Bernadette Kent, Instructional Courseware Designer, Tertiary Education Institute - CAL, University of Queensland, 4072, Brisbane, Australia. Phone: (07) 365 1916; Fax: (07) 365 1799; Email: b.kent@mailbox.uq.oz.au
Ann Russell, Instructional Courseware Designer, Tertiary Education Institute - CAL, University of Queensland, 4072, Brisbane, Australia. Phone: (07) 365 2543; Fax: (07) 365 1799; Email: a.russell@mailbox.uq.oz.au
Please cite as: Dymalla, A., Kent, B. and Russell, A. (1994). Towards successful computer based learning - the complete process. In J. Steele and J. G. Hedberg (eds), Learning Environment Technology: Selected papers from LETA 94, 57-62. Canberra: AJET Publications. http://www.aset.org.au/confs/edtech94/ak/dymalla.html |
[ EdTech'94 contents ]
[ ASET Confs ]
This URL: http://www.aset.org.au/confs/edtech94/ak/dymalla.html
Last revised 24 May 2003. HTML editor: Roger Atkinson
Previous URL 17 May 1999 to 30 Sep 2002: http://cleo.murdoch.edu.au/aset/confs/edtech94/ak/dymalla.html
