Full Paper

Rodney J. Clarke

rodney_clarke@uow.edu.au

Lawrie Schafe

Lawrie_Schafe@uow.edu.au

Department of Business Systems, University of Wollongong

Introductory tertiary information systems subjects often focus on the information systems and technologies separated from the organisational contexts which motivate their use. The purpose of the ëInformation Systems in Contextí CAUT project (Schafe and Clarke 1995, 1997a), was to show how a range of information systems and technologies are used to support work processes in real business settings. The development of this project was technically challenging. The project required extensive communication between several disparate groups (including analysts, designers, artists, stakeholders and users). The project also involved a large number of media types and production techniques. Some media types required extensive field trials, while some production techniques, novel to this project, required proof-of-concept studies conducted in parallel with system development. In common with most multimedia titles, this system had all the hall-marks of a high risk systems development project. The risks associated with systems development projects of this type can be reduced by using development techniques based on prototyping. This paper describes the use of one type of prototyping referred to as evolutionary systems development (ESD). ESD was employed to ensure that an appropriate system, one which served the needs of the all the stakeholders, was built.

Educational multimedia projects are fraught with risks, anyone of which could seriously compromise the completion of a project. The characteristics of the ëInformation Systems in Contextí project which predisposed it to development risks are described below. Major project risks involve establishing an appropriate scope for the project, identifying and maintaining communication between stakeholders during the life of the project, and in gaining experience needed to deal with disparate media types. Production techniques to handle diverse media types may need to be created by the project team, in parallel with the development of the multimedia itself.

After a lengthy negotiation period with a number of companies, it was decided that ìSupply and Warehousing at BHP: Stocked Repairable Itemsî would be used as the basis of the case study. The scope of the project was constrained to repairable items stocked at a central warehouse (Illawarra Distribution Centre or IDC). The Supply Process spans many business units at BHP including internal customers who generate requests for items from the IDC as part of regular maintenance, and the Computing Centre where, amongst other things, requests for stocked repairable items are processed. The determination of a realistic project scope is extremely important for multimedia titles. Too large a scope will exhaust project funds, or worse the project team will become daunted and the project will never be completed. These risks are multiplied for complex case studies of real organisations. Setting a very constrained scope early in the project proved to be a wise development tactic.

In common with other multimedia projects, this project required coordination and communication between multiple development teams. These teams included the Project Team consisting of the grant recipients, as well as several Developer Teams including an interactive multimedia unit, a video production unit and a text production unit. When separate development teams are used, as was the case with this project, it becomes necessary to frequently discuss the status of the project and its progress. Despite the best of intensions and the highest project moral, programmers and artists often have serious constraints on their time. Continuous co-ordination between the Project Team and the Developer Teams, instilled in the latter the importance of the project.

Despite the requirements for, and usefulness of, feedback from a Reference Group, it was often extremely difficult to conduct regular meetings. The more ëhigh-poweredí and inclusive the Reference Group, the more difficult it was to meet as a group. The strategy used in this project was to set up a Developmental Website, which included detailed information concerning the design of the system. The website was useful for several reasons: (i) it became a repository for the current specification of the system, (ii) it formed a historical record of the changes in the system over time, and (iii) it enabled an external Advisory Group, consisting mostly of overseas experts, to be kept up-to-date with developments. Despite its utility, the use of a Development Website to substitute for frequent Reference Group meetings had some disadvantages. It transformed the kinds of communication between the Project Team and the Reference Group, from the interaction typical of these meetings to asynchronous interactions. As a consequence, the brainstorming and idea generation which is a characteristic of Reference Groups in multimedia projects, was limited.

As well as communication and coordination within and between the Project Team, the Development Team, the Reference Group, and the Advisory Group, systems development projects occasionally reveal unanticipated or emergent stakeholder groups. It soon became apparent that one group of stakeholders, the management and workers at the Sites being studied, had not been considered when the project was proposed. The Site workers and management were none-the-less critical to the success of the project. With the benefit of hindsight, the Project Team should have anticipated this. Excluding these stakeholders would have been catastrophic for this project. In order to include the Sites into the project, special meetings were established. The relevant Site managers and workers were given the opportunity to view and exclude any material collected at their sites (for example, video footage). The full inclusion of the sites managers and workers into the project, helped to establish a high level of trust between the Project Team and the various Sites. The Project Team was permitted the kind of ground-level access about which most Systems Analysts can only dream. The Site management and workers endured work slippages and very high levels of intrusion into their workpractices, in order to assist in a project which had become relevant to them.

This project was technically challenging, both in terms of the media types used and also the production techniques developed in order to fabricate the multimedia system. As a consequence, the Project Team needed to undertake several small exploratory prototypes and proof-of-concept studies. Three examples are described below, the first relates to a media type, while the other examples involve production techniques.

• Given that a number of partially immersive Virtual Reality (VR) were to be used in the project as frameworks for other media, an exploratory prototype was of vital importance. This was especially the case given the lack of published experience in the educational uses of this technology (see Schafe and Clarke 1997b). The construction of this exploratory prototype enabled the Project Team to select equipment, check on the best photographic emulsion, test lighting conditions, and establish a useful field protocol, amongst other considerations.
• Early in the project, several exploratory prototypes of the user interface were created. The user interface was a novel aspect of the project, and was designed using an unusual metaphor. As the multimedia system was to be used both inside the university and also at various sites within BHP, the user interface was designed to have as few widgets as possible. The user interface consists of only three main widgets: (i) a navigation widget, (ii) a case study widget and (iii) a safety gear/security badge widget. By far the most complex widget in terms of design was the navigation widget. An artificial character called the kinetic actor, stands in the centre of a circular stage. The stage consists of labeled pie segments, each of which represents an option. Moving the mouse around the stage, makes the kinetic actor move to face an option. Clicking an option makes the character jump to a selected option. The hemisphere of options directly ahead of the character represent options that have not yet been visited, while the hemisphere of options directly behind the character represent options that have been visited by the character (and the user). The exploratory prototypes suggested the best way of implementing the widget, assisted in the creation of a standard production technique for assembling various multimedia segments, and helped create standard aesthetics for the project.
• The project made extensive use of genre theory in systemic-functional linguistics to organise the projectsí textual media. In fact multimedia equivalents of canonical genres were developed, including a multimedia case study genre and variants on a number of factual genres. A proof-of-concept study was made of the ëwritingí practice associated with converting text to hypertext (Clarke 1997a). Genre theory was also used as a way of converting, ërawí interview texts conducted on site into appropriate text chunks in the multimedia system. This aspect of the project also required a proof-of-concept study (Clarke 1997b).

The use of exploratory prototypes and proof-of-concept studies conducted in parallel with normal systems development activities formed a risk aversion strategy which enabled the Project Team to determine difficulties and costs early in the development phase. Technically demanding initiatives involving high levels of complexity, innovation and new technologies, where the requirements are subject to change over the course of the project, and which demand frequent communication between disparate groups, have all the hallmarks of high-risk systems development projects. The risks associated with systems development projects of this type can be reduced by using development techniques based on prototyping.

While a prototyping approach to complex systems development is well established in the IT/IS and engineering sectors (Crinnion 1991), its use in the development of educational multimedia is less familiar. Of the various types of prototyping, Evolutionary Systems Development (ESD) arguably provides the most appropriate model for educational multimedia development. ESD was utilised as the model of systems development adopted in this project. It provided a number of conceptual and practical advantages over traditional forms of systems development, and in the case of this educational multimedia development reduced the risks identified above.

In an ESD project, the problem is broken into a number of prototyping cycles. Each prototyping cycle solves a single aspect of the problem and consists of three phases. The first phase is called the develop phase in which a computer-based model of a sub-problem is built. Once the first phase is complete, a second phase called the test phase is undertaken. The computer model is tested against either (i) previously established explicit design criteria, or (ii) through stakeholder evaluation. Once the second phase is complete, an amend phase is undertaken. The amend phase involves implementing those changes suggested in the previous phase. The prototype cycle is successively iterated through all phases until the sub-problem is completely implemented, see the insert in Figure 1. Occasionally, a prototype reveals that there is more than one sub-problem to be solved. If each of these sub-problems are non-trivial, that is the problem is more complex than initially thought or the scope of the project needs expansion, then two or more prototyping cycles are spawned, one for each sub-problem (black lines). Alternatively, two or more prototyping cycles may be merged together if a given problem was simpler than initially estimated (grey lines).

ESD stands in stark contrast to traditional systems development approaches which dictate that all major project requirements be known in advance of any production. Unlike traditional systems development, once the project scope has been established the system is broken up into a number of relatively independent subsystems. In the case of this project, the multimedia system was initially separated into three prototypes, one for each of the sites. As the IDC Warehouse was the site which required the greatest attention and development effort, its prototyping cycle was started earlier and iterated faster than the others. Technical solutions developed in this prototype provided other prototypes with solutions. In other words, technical solutions developed once were used many times. ESD also accommodated changes to system requirements. For example, Site management suggested that the project include information about site safety and security access procedures. The Project Team built this into the user interface by providing a widget which showed either safety clothing or a security badge. The kinetic actor gestures towards this widget when necessary, and does not proceed until the user has clicked it. Once the user has clicked the widget, the kinetic actor acquires and uses the appropriate items.

ESD is a ëbottom-upí development model, which consequently reduces the risks associated with traditional ëtop-downí development models that force Project and Development Teams to make decisions early in the project- at the point of greatest ignorance. ESD privaledges the systems development process, rather than the products of the development process. The systems development process emphasisies communication between participants. Consequently ESD can assist with situations where either emergent stakeholders, whose interests cannot be ignored, become apparent, or when ësatisficingí the interests of multiple sets of stakeholders whose interests may be in conflict.

ESD not only assists educational multimedia developers prior to acceptance (systems development) but also post-acceptance (systems use). Traditional systems development has a single fixed acceptance stage, where the product is considered complete. In fact, the product simply changes its status from being ëunder developmentí to being ëmaintainedí. In contrast to direct cutover in traditional systems development, ESD blurs the distinction between development and acceptance to form what is referred to as incremental delivery. Incremental delivery means that the important prototypes are developed and delivered early; less important components are developed latter. In the ëInformation Systems in Contextí project, incremental delivery was used in two ways. First, the project was developed through a series of three functional versions (Version 1: Warehouse; Version 2: Computer Centre plus the final version of the Warehouse ; Version 3: Final iteration including a site which constituted a Customer for the business process). Second, the multimedia system, while primarily centred on the creation of a CD-ROM, also utilised a Production Website where additional media could be added as needed in a post-acceptance stage.

Interesting methodological and substantive affinities (Burgess 1982) exist between evolutionary prototyping of the kind advocated here for the development of multimedia in educational environments and action research commonly used in pedagogic applications (Zuber-Skerritt 1992a, 1992b). Figure 2a shows the steps involved in undertaking an action research project. Action research projects often take the form of a preparatory sub-sequence of steps (Initiation: Hypothesis), and a concluding sub-sequence of steps (Dissemination: Follow-up). In between these preparatory and concluding sub-sequences is the so-called action research spiral which iterates a sub-sequence of Intervention: Evaluation. It is this spiral of steps which characterises an action research project. The steps involved in action research are compared to those of ESD in Figure 2b. By illustrating the affinities between evolutionary prototyping and action research, educational multimedia developers may become more convinced of the utility of ESD over traditional forms of systems development. These affinities await further research in order to determine if (i) there are contributions to evolutionary prototyping that can come from an undertstanding of action research, and conversely (ii) if there are contributions to action research that can come from an understanding of evolutionary prototyping.

Educational multimedia development projects like the ëInformation Systems in Contextí CAUT project, have all of the characteristics of high-risk projects. This paper describes the risks associated with this project, and advocates the use of ESD as a means of containing them. Arguably, it is these project risks which contribute to the relatively low completion rates of educational multimedia projects. ESD has characteristics which are similar to those of Action Research. Consequently educational multimedia developers may be more comfortable with ESD compared to traditional systems development models which are known to be a risk prone.

(c) Rodney J. Clarke & Lawrie Schafe

The author(s) assign to ASCILITE and educational and non-profit institutions a non-exclusive licence to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. The author(s) also grant a non-exclusive licence to ASCILITE to publish this document in full on the World Wide Web and on CD-ROM and in printed form with the ASCILITE 97 conference papers, and for the documents to be published on mirrors on the World Wide Web. Any other usage is prohibited without the express permission of the authors.