|
This paper describes a systematic approach to the design and development of interactive multimedia courseware for language learning. Critical management issues which affect the project's success at each stage of the development process are discussed. The complex nature of multimedia development is explored both in terms of the roles and responsibilities of team members as well as the interrelationships of activities at each stage of the development process, from analysis and design, through to development, implementation and evaluation. Recommendations for future development efforts include ensuring that adequate resources including time, money and staff expertise are provided, and that ultimate authority for team management is clearly vested in the project leader. In addition, the project leader's understanding of Instructional Systems Development methodology is stressed.
The purpose of this paper is to discuss management issues which are critical to the success of interactive multimedia courseware projects. These issues have emerged as significant events in the development of computer aided language learning (CALL) materials for a University course to be delivered in the distance learning mode. The author's perspective is that of instructional designer and project leader of a 5 person development team, which was established in the spring semester 1991. Some of the issues raised here are especially pertinent to large scale development projects which occur in academic settings.
The design and development of multimedia courseware is a complex process which requires the expertise of a range of professionals, including content specialists, graphic designers, audio-visual production specialists, computer programmers, and instructional designers, just to name a few. In addition, each of these professionals has their own ideas about education and training. Thus multimedia project leaders must be skilled in managing the interdependent tasks in the development process, as well as in facilitating a team effort. One of the most effective ways to manage this complexity is to use Instructional Systems Development (ISD) methodology.
The ISD model is applied by various authors to either macro or micro level design problems. At the macro level ISD is used to guide the development of complete instructional systems or training programs, while at the micro level it can be applied to the design of an individual lesson on a given topic. Accordingly, this leaves much room for variation in actual application, depending on the specifications of the individual project.
This paper presents issues that have emerged in the process of applying the ISD model to multimedia development. ISD was used to guide the design and development of computer aided language learning (CALL) courseware. (The CALL courseware described here can be classified as "multimedia" in that the final product incorporates a range of media, including text, graphics, animation, and sound.) ISD methodology was chosen to ensure high quality yet cost effective outcomes for each of the five stages in the development process - analysis, design, development, implementation and evaluation. While use of this methodology will not guarantee success, it is nevertheless a useful approach for multimedia materials where development costs are generally far greater than the production costs associated with the development of the component parts. Therefore, the following discussion will be presented in the context of the stages of the ISD model.
In the case of the CALL project described here, the performance or teaching problem had been identified by the Department of School Education - a shortage of secondary language teachers in the North Coast Region of NSW with the skills to teach Japanese. Moreover, many of the currently employed language teachers were (and still are) interested in extending their skills to include the teaching of Japanese, but they could not take time off work to attend a University course located in a distant city. Thus, they needed a University course, offered in the distance learning mode, and designed to match the secondary Japanese language curriculum guidelines in the New South Wales schools. Further investigation revealed that this shortage existed across the state of New South Wales and indeed across the country. Consequently, the target population for this course seemed to be potentially quite large.
Preliminary investigations into the feasibility of this project by upper level management at both the University of New England, Northern Rivers and the Department of School Education proposed that recent developments in computer technology, such as CD-ROM, be used to deliver the instruction. They hypothesised that the new technology had immense storage capability, as well as the ability to record speech and play it back. Moreover, they felt that the need for Japanese language instruction offered in a distance learning mode was so great, that the demand and thus the market value of the final product, would offset the initially high development costs.
It was in this early formative stage of the project that a thorough review of the literature on computer aided learning systems for teaching Japanese was commissioned. The reviewers, Claff, McNutt & Wilks (1989), found that language learning had proven to be a learning task well suited for computer aided learning. CALL programs in which digitised speech is used to provide a model of the native speaker's pronunciation, and which provide the learner with the ability to record his or her own voice were considered the most effective. (See Lian, 1991, for further details).
Thus, although the teaching or performance problem had been identified and the initial review of CALL programs was favourable, additional information about the learners and the environment in which they would be using the instructional materials was still needed. Therefore, the instructional designer proceeded to conduct an environment analysis, which is a particularly useful analysis for multimedia and CBT development in general. Tessmer (1991, p. 9) explains that environment analysis is "the analysis of the context in which the instructional product will be employed, of the physical and use factors of the instructional environment and its support environment." A detailed investigation of these factors, as described below, provided the rationale for later design decisions.
In this project, the focus of the environment analysis was on the target group and the way(s) in which they might use multimedia materials. The target group comprised teachers employed in secondary schools in the North Coast Region. These schools were supported by five educational resource centres (ERCs), which were geographically dispersed throughout the region, such that each school was no more than a one hour drive to its nearest ERC. While it was not realistic to expect every teacher to have a computer at home, the presence and location of these ERCs made it feasible to consider a multimedia solution to the training problem. The teachers could use a combination of printed workbooks, audiotapes and computer aided language learning (CALL) materials. The workbooks and tapes would be available to the teachers at all times and could be used at home, at work, in the car, or at the ERCs, wherever they chose to do their studies. On the other hand, the CALL lessons could be accessed on specially equipped computer workstations installed at each ERC. In fact, development decisions were later based on the assumption that an Apple Macintosh LC computer equipped with headphones, a microphone, voice recording software (Macrecorder), and an external drive (a 44 MB cartridge drive, and later CD-ROM drives) would be placed in each of the five ERCs in the region.
While the environment analysis had shown that computer delivery of course content was feasible, and the review of the literature supported CALL development, additional information on the specific learning requirements for the course was needed before the choice of instructional strategies could be finalised.
Thus, the output of the analysis stage in this project included the identification of the instructional goal or teaching/learning problem, the review of the literature on CALL, and the feasibility data from the environment analysis. This information contributed to the important design decisions which had to be addressed in Stage Two of the development process.
In this project, the broad learning goal was to create instruction which fostered a communicative approach. This approach includes the ability to read and write words, phrases and sentences in all three Japanese scripts (Hiragana, Katakana and Kanji), as well as to communicate orally (ie. to develop speaking and listening skills) in Japanese. These requirements along with the data collected in the analysis stage led to the selection of interactive multimedia as the best medium to achieve the learning goals. Multimedia, with options of playing and recording digitised speech, displaying high quality graphic images (including some animation), and offering a variety of text options for showing both Japanese script and the English equivalent, seemed to satisfy all learning requirements with the exception of the writing function, which easily could be addressed in the accompanying print materials. The next step was to consider lesson design options for the multimedia materials.
A critical issue for instructional designers at this stage is to explore innovative ways of delivering the content. This is especially true for the design of CAL and multimedia materials. Because instructional designers are skilled in instructional analysis, or breaking down more complex learning tasks into smaller subtasks and translating these subtasks into behavioural objectives, it is tempting to then design an instructional sequence which provides for step by step mastery of these subtasks. However, this approach can result in boring, linear programs in which the emphasis is on mastery of isolated skills rather than the synthesis of skills, or application level learning.
A better alternative is to experiment with more dynamic designs, which can be programmed in a prototype version and trialed with learners from the target audience. Tripp and Bichelmeyer (1990, p 38) suggest that instructional designers use the software design methodology of "rapid prototyping" to experiment with new ways to facilitate learning with CBT and multimedia materials. These authors recommend this approach be used whenever designers are working with development tools that offer both modularity and plasticity (computer software such as HyperCard, for example). Moreover, they suggest it is a useful approach in "new situations where there is not an abundance of experience from which to draw." It is also extremely valuable in situations where development costs are high, as in the production of multimedia materials.
For all of these reasons, a prototype CALL program was created in the early days of this project, using HyperCard as the development tool. The design team produced an interactive learning environment in which learner control was preferred over system control, and learners had the opportunity to explore a number of different paths through the program. In addition, a number of different modes including tutorial, drill and games, for example, were incorporated into the one CALL package. It was subsequently trialed with teachers from the target audience. As a result, considerable changes in the design were made prior to commencing full scale development of the CALL component of the course. Thus rapid prototyping afforded the opportunity to experiment with a new design and to collect valuable data on its effectiveness prior to full scale development.
Effective instructional systems development projects require careful management throughout the whole process. Without such careful oversight and coordination, projects fail. Some of the needs for management are not at all unique to ISD projects; any time there is a group of people working on a variety of tasks over time, there is likely a need for management. When the tasks are interdependent, as in instructional systems development projects, then the need for management is even greater (p. 157).In his book Computer Based Training: A Handbook for Training Professionals, Sims (1991) explains that the duties of the project manager include negotiating with upper level managers and clients to define the scope of the project, developing standards, identifying potential problem areas, procuring funding, setting up the project team and making appropriate work assignments. Of these duties, one of the most important is the selection of team members. Because multimedia development requires many different kinds of expertise, the manager must consider whether s/he will hire new people, train existing staff, deploy trained staff from other departments, or retain consultants. In some cases one person will have expertise in more than one area, but the manager must decide which skills are needed at what time and make the appropriate work assignments. When roles and responsibilities for team members are clearly identified at the outset, and a climate of mutual respect and open communication is established, then the potential for interpersonal conflict among team members is greatly reduced. Nevertheless, new team members must be briefed on the interdependent nature and resulting pressures of working on a development team.
In the CALL development project described here, the team members included the instructional designer and project manager, a full time course writer (content specialist), a full time graphic artist, two full time computer programmers, and part time contract help to assist with word processing and the audio recording of native speakers. The course writer and graphic artist were hired for their skills in each of these areas, while the computer programmers were deployed from another part of the organisation and trained in the use of the authoring system that was used. The instructional designer/project leader was the only team member who had worked on a CBT or multimedia development team previously. Therefore, a considerable amount of time was needed in the beginning of the development stage to orient team members to their job tasks.
As noted above, the instructional designer on the CALL project also served as the project manager. There were advantages and disadvantages to this arrangement. However, Hannum & Hansen (1989) suggest that instructional designers have the necessary education and design skills, as well as familiarity with a range of media to manage the development process. These authors believe that instructional designers, and not content specialists, should direct ISD projects.
It is essential to the success of a project that the designer have the required authority and leverage to make ISD decisions; that is, decisions driven by instructional principles. Of course, content persons and ISD specialists can discuss how to go about some task, like whether to use lecture or computer based training to teach a certain objective. However, when it is time to make the decision and move forward, someone must be empowered to say 'This is the way it is going to be.' that person should be a skilled designer, not a content person. Rarely is there time for content persons to learn enough about ISD to function in this role (p 158).This author has noted that in tertiary institutions in Australia, this is often not the case. It seems that it is often the academic content specialist who has the final word in design decisions.
Thus, it is imperative that the project manager not only have the authority to make design and development decisions, but s/he must also have adequate resources to complete the job on time and within budget. These resources include sufficient allocation of staff time to work on the project. One of the difficulties which persisted throughout the first year of development in the CALL project described here was the restriction on staff resources imposed by management which limited the instructional designer/project leader's involvement in the project to only 50% of her time.
Sims (1990, p 49) explains that it is the project manager's responsibility to ensure that "resources are available when required, and to obtain additional resources if required." He further notes some of the difficulties in estimating time and resources required for development tasks. The author (Sims, 1990, p 49) explains that the manager must realise that "an activity which is estimated to take one author 40 hours to complete will not necessarily be halved if two authors are allocated to the same activity." This was an assumption which was made in the CALL project described here. Two programmers were assigned to the project to speed up the development process. However, the resulting complications with job task allocations, additional computer hardware and software needs, and other procedural details did not result in a significant reduction in development time. In fact, periodic imbalances in the work flow could be attributed in part to this team arrangement.
To summarise, Hannum & Hansen (1989) identify a number of ways in which ISD projects can "go astray."
When there is an adequate staff following an acceptable model in completing an ISD project the remaining area associated with failure is inadequate management. If resources are not made available or are misused, if schedules are not kept, if the quality of the output from each step in an ISD model is not monitored, if effective communications are not the norm, or if a spirit of mutual respect and cooperation among the staff is missing, instructional systems development projects can go astray and be very unproductive (p 159).These authors (Hannum & Hansen, 1989) suggest that project managers must pay particular attention to two areas: (1) project management and control, and (2) consulting/negotiating tasks. In the first area, the manager must be able to estimate the amount of time and resources needed to complete each step of the ISD process, to schedule the work necessary to complete the project, and to monitor the work and make necessary adjustments so the project can be completed on time and within budget. In the second area, the manager must be able to establish and maintain smooth working relationships among the team members. It is perhaps worth emphasising that in addition to expertise in the skills required for design and development work, the criteria for team membership must include good communication skills and an ability to work well as a member of a team. These are words often seen in job advertisements, but they describe a quality which is quite difficult for the project manager to assess ahead of time.
The next two stages in the development process, implementation and evaluation, have not officially begun in the CALL project described here. However, some issues related to each of these stages have been identified by the project manager in anticipation of work required in these areas. Thus, overlap in the stages of the ISD model is an issue that future project leaders must be prepared to address.
In the CALL project, studio recording sessions with native speakers were conducted only twice in the first 12 months of the project. For the rest of the time, draft quality recordings were made by the course writer and incorporated into the courseware by the programmers. Similarly, the programmers authored the courseware on computers and saved copies of the courseware on external cartridges, which could be easily updated. These cartridges would be used in the first stage of implementation, so changes could be made prior to pressing a CD-ROM disk.
Another consideration in developing materials in an academic setting for tertiary courses is the conflicting roles of the content specialist during implementation. If the content specialist is a full time faculty member, his or her time for development work (ie. writing and advising on media integration) will very likely be quite limited. However, an advantage is that this person will have a better understanding of how the new material relates to material in other courses offered in the Department, as well as an understanding of the procedural and assessment requirements that must be addressed when implementing these materials in a University course. On the other hand, even when the content specialist has been hired as a full time team member, complications arise when s/he is needed for full time development work in the development stage, as well as for assistance with the implementation and evaluation activities. Because in most cases, multimedia development work on the next set of materials will continue while the implementation and evaluation of the first set of materials is taking place. It is best to address these problems in the early stages of team selection and orientation, rather than waiting to find additional content specialist help at the implementation and evaluation stages.
Sims (1990) suggests that project managers must consider the following questions to guide the management of evaluation efforts:
What will be evaluated?In addition, the manager must decide who will conduct the evaluations - an external reviewer/content specialist? an external instructional designer? an external computer specialist? the client or customer? a member of the target audience? Each of these individuals' perspectives will provide valuable evaluation data.How will evaluators communicate their comments?
When do evaluations need to take place? and
What is the process for making revisions? (p 76)
Once the evaluation data is collected, the project manager must consider all recommendations and other factors, then decide which aspects of the program to revise. As Sims (1990) explains, the manager must determine "which evaluation comments indicate problems in the courseware, prioritise those problems and make revision decisions, ensuring that staff are available to make the specified revisions. The extent to which revisions can be implemented will depend on the impact on the project completion date (p. 76)."
In the CALL project described here, revision decisions had to be made continuously. As a guiding principle, the evaluation data from the learners should dictate which changes are absolutely necessary. However, many revisions must be made prior to the evaluation stage. The iterative nature of the development process necessitates revising earlier efforts. In fact, the project manager is evaluating the quality of the product continuously, as s/he monitors the work of the team. However, it can become a sensitive issue for team members who have revised the same material several times already and feel that any further changes are a waste of their time. Therefore, these quality control decisions must be made in light of any negative effects they may have on team morale.
Instructional Systems Development methodology was proposed as a useful tool to guide the development process. In addition, the complex nature of multimedia development was explored. Two factors which contribute to this complexity are the interdependencies among team members, and the interdependencies, or overlap, in the activities which occur in each of the five stages of the ISD model.
Thus successful multimedia development projects are those in which the project manager is skilled in Instructional Systems Design methodology, where adequate resources have been allocated so the project can be completed on time and within budget, and where ultimate authority for team management has been dearly vested in the project leader.
Bergman, R. E. & Moore, T. V. (1990). Managing Interactive Video/Multimedia Projects. Englewood Cliffs, NJ: Educational Technology Publications, Inc.
Claff, J., McNutt, L., Wilks, B.(1989). Technology report: Computer-based courses in Japanese as a second language. Lismore: University of New England, Northern Rivers, Learning and Information Centre.
Hannum, W. & Hansen, C. (1989). Instructional Systems Development in Large Organizations. Englewood Cliffs, New Jersey: Educational Technology Publications, Inc.
Lian, A. (1991). Applications of interactive digitised audio technology in language learning. Essays in Honour of K. V. Sinclair. Townsville: James Cook University.
Sims, R. (1991). Computer Based Training: A Handbook for Training Professionals. Mona Vale, New South Wales: Knowledgecraft Pty Ltd.
Tessmer, M. (1991). Back to the future: The environment analysis stage of front end analysis. Performance and Instruction, 30(1), 9-12.
Tripp, S. D. & Bichelmeyer, B. (1990). Rapid prototyping: An alternative instructional design strategy. Educational Technology Research and Development, 38(1), 31-44.
| Author: Dr Patricia Youngblood is a Senior Lecturer in the Faculty of Education, Work and Training at the University of New England, Northern Rivers. She teaches both undergraduate and postgraduate level courses in Training and Development and Instructional Design. In addition, she has worked as an instructional designer in both the academic and commercial sectors in the United States, and has directed CBT and multimedia courseware development in medical education, management training and language learning for teachers. She can be contacted at the Faculty of Education, Work and Training, University of New England, Northern Rivers, PO Box 157, Lismore NSW 2480
Please cite as: Youngblood, P. (1992). Issues in the development of multimedia courseware. In J. G. Hedberg and J. Steele (eds), Educational Technology for the Clever Country: Selected papers from EdTech'92, 93-101. Canberra: AJET Publications. http://www.aset.org.au/confs/edtech92/youngblood.html |