International
Journal of
Educational
Technology

International computer-supported collaborative teamwork in business education: A case study and evaluation

Jane E. Klobas, University of Western Australia
Gaby Haddow, University of Western Australia

Abstract

This article describes how a computer-supported collaborative learning project, conducted across several business schools in different countries, was evaluated by the instructors at one of the participating schools. The students worked in virtual teams to produce a common project outcome, a business plan. The evaluation was based on: learning outcomes as measured by team project scores, individual scores on learning about the topics the collaboration was designed to illustrate, and student reports of the relationship between participation and learning; student satisfaction with experience as team members and with the process of learning through participation in the exercise; and statistical analyses of the relationship between learning and participation. The goals for the project were met: most students explicitly related their learning about working in virtual teams to participation in the collaboration rather than to other course activities, and most students were satisfied with this approach to learning. The authors attribute the success of the project to the way it was integrated into course design. They suggest that learning about virtual teamwork could be attained to the satisfaction of more students if more project time were spent on team building and less on tasks associated with a complex project.

A recurrent theme in organisational design throughout the 1990s has been the use of teams to achieve greater levels of performance on tasks: "teams and good performance are inseparable: you cannot have one without the other" (Katzenbach & Smith, 1993). In parallel, there has been considerable discussion of the role of computer-supported communication technologies in supporting and enhancing the work of teams (Jarvenpaa & Ives, 1994; Nohria & Eccles, 1992). Networked communication technologies have the potential, if used appropriately, to improve co-ordination among members of project teams (Allen & Hauptman, 1990; Gorton & Motwani, 1996; Keen, 1990).

Members of computer-supported dispersed teams face many challenges, however. They must communicate the detail and the nuances of much communications in written text, without the assistance of gestures and non-verbal cues (Sproull & Kiesler, 1991). Members of internationally dispersed teams may not share a common first language or business culture (Davison, 1994; Henry & Hartzler, 1997).

The combination of team work and enabling communication technology has the potential to benefit management education as well as management practice (Alavi, 1994; Alavi, Wheeler & Valacich, 1995; Ives & Jarvenpaa, 1996). Collaborative theories of learning propose that students learn effectively through participation in exercises that permit them to learn from one another (Johnson & Johnson, 1989; Kimber, 1996). In management terms, this would be described as a 'team' effect. A growing body of theory supports the use of computer-supported collaborative learning for graduate business education, where students are adults who respond well to active, co-operative and problem-based learning environments (Alavi, 1994; Angehrn & Nabeth, 1997; Leidner & Jarvenpaa, 1995).

International collaborations have the potential to introduce students to business mores and conditions in other countries. Using networked technologies, students from different cultural and language groups can work together on joint projects regardless of geographical location or time zone. The tasks set for business students working in dispersed teams can simulate the types of task they may meet in the 'real' business world. Thus, if effective, computer-supported collaborative learning in international virtual teams can have two principal benefits for students. Firstly, students can learn about the set task and topic through collaboration with a diverse group of peers. Secondly, they should be able to learn about the process of working in a business environment which incorporates dispersed computer-supported work teams.

A Global Virtual Team Collaboration

In 1993, Professor Sirkka Jarvenpaa and colleagues at the Graduate School of Business at the University of Austin, Texas, initiated the Global Virtual Team (GVT) Collaboration. The GVT Collaboration provides an opportunity for advanced business and information systems students to learn about computer-supported work in internationally dispersed teams. Students from universities around the world work together in teams of up to 6 to complete a set task, such as preparation of a business plan for an Internet-related business.

Jarvenpaa and her colleagues have used the GVT Collaborations to study several aspects of global virtual teamwork (Jarvenpaa, Knoll & Leidner, 1998; Knoll & Jarvenpaa, 1998). They have identified team behaviours which can build the social skills associated with effective project completion, good team scores on the set task, and satisfactory student experience. Other researchers in the field have focused on information transfer and other processes associated with building teams (Cramton, 1997) and some recent case studies have included observations about how to make the experience more satisfactory for students and instructors (Hazemi, Hailes & Wilbur, 1998; Bielli, Renzi, Kumar & Klobas, 1999). Overall, though, little attention has been paid to evaluation of learning outcomes.

This article therefore describes how one participating university evaluated outcomes from students' participation in the 1997 GVT Collaboration. The research addressed the question, "Do business students learn from participation in collaborative teamwork among members of geographically dispersed teams?". We also examined student satisfaction with this approach to learning.

Methods

Student learning from, and satisfaction with, the GVT Collaboration was studied with the co-operation of MBA students at the University of Western Australia (UWA). In this section, we describe how the GVT Collaboration was incorporated into the MBA at UWA, how evaluation criteria were established, and how the evaluation was conducted.

The Fifth GVT Collaboration at the University of Western Australia

At the University of Western Australia (UWA), the GVT Collaboration is run as part of the optional core unit, Information Management 560 (IM560). The fifth GVT Collaboration was run over 10 weeks from February to April 1997. At the end of this period, each global virtual team was required to produce a business plan for an Internet-based international consultancy specialising in integrated software. The teams were provided with a template for the business plan, but had to apply their coursework, reading and research to prepare the content of the plan.

Through their participation in the 1997 GVT Collaboration, we expected UWA MBA students to learn

• about integrated software packages,
• about the potential for global networked organisations, and
• effective behaviours for computer-mediated communication in a geographically-dispersed project team (the process of global virtual team work).

The course included a session on the role of integrated software packages, and guest lectures from representatives of each of the three main Western Australian providers of integrated software. The theory of global networked organisations was covered through readings, an electronic discussion of the readings using Lotus Domino software, and a face-to-face session in class. Guidelines for effective behaviour in computer-supported teams were introduced after a workshop in which students learnt to use Lotus Domino for discussion. The GVT Collaboration was expected to provide a realistic and satisfying experience through which students would consolidate their learning about these topics.

All 56 students enrolled in IM560 in 1997 elected to participate in the GVT Collaboration. After their final examination was graded, all students were asked if their course submissions and grades could be used in an evaluation of global virtual teamwork. Forty-five (45) students responded to this request in time to be included in the evaluation. This group represents 90-95% of students who could be located at that time. (Five non-respondents had completed their MBA and left the university, and several others were travelling on vacation.) Three respondents were excluded: one declined to be involved, and two others had no prior work experience and were therefore not typical of the School's body of MBAs. Of the 42 students included in the research, 25 (59.5%) were in wholly virtual teams (all members were working at different, geographically dispersed, universities) and the remainder were in teams that included two or more local UWA participants.

The participants were fairly typical of students enrolled in UWA's MBA. Their ages ranged from 26 to 50, with an average age of 35 (a little older than the norm for the degree). Most were taking the course in evening sessions while working full-time. The participants were from a range of ethnic, cultural, language, and business backgrounds. Their home countries and places of work included Australia, Canada, China, England, India, Indonesia, Japan, Malaysia, and Singapore. Their fields of work included accounting, banking, computer networking, export consultancy, hospital management, human resource management, physiotherapy, property sales and management, and public sector management. A slightly higher proportion came from non-technical (57.1%, 24) than from technical (42.9%, 18) fields.

Before commencing the GVT, UWA students were required to complete a series of practical exercises to ensure they had the necessary skills as Internet users to participate effectively in the project. By the time they had completed the Internet skills component of the course, there were no observable differences in the measured basic Internet skills of prior users and non-users (t for difference between users' and non-users' Internet skills scores = 1.17, df = 39, p> .05).

To support their work in the GVT, each team had its own listserv, administered by the University of Texas, for distribution of electronic mail messages. Although some teams experimented with fax, Internet chat and World Wide Web services to supplement their email communication, most communication was by email (with attachments as appropriate) through the listserv.

Evaluation criteria

Learning outcomes were measured by instructor-assigned scores on the relevant assessable components of the course. Student satisfaction with learning was explored through analysis of students' reports of their experiences as team members and supplemented by students' remarks about the course as a whole. The extent to which learning was associated with participation in the GVT was studied through the extent to which students explicitly linked their learning to their experience in the GVT and through statistical analyses of differences in learning and participation. Measurement of each criterion, and of the relationship between learning and participation, is described in this section. The variables are summarised in the Appendix and discussed in the following sections.

Instructor-measured indicators of learning

Two instructor-measured indicators were used to gauge learning: project score, and topic learning. Project score is an indicator of how much the team, as a whole, learnt about the topic. Individual learning about integrated software, dispersed teamwork and organisations, and effective behaviours in virtual teams was measured by marks earned on examination questions designed specifically to measure learning about these topics (topic score). The questions (see Table 1) were designed to test deep learning; it was impossible for a student to pass the exam simply by repeating words or models drawn from a text or their reading for the unit.

Table 1. Examination questions on which topic learning score was based

Student satisfaction and learning through participation

Student satisfaction with the GVT, and the perceived relationship between participation and learning, were measured through textual analysis of examination scripts and responses to independent evaluations of the course as a whole.

The textual analysis was completed by a research officer who was not a member of the School's teaching staff to reduce the possibility of bias. (The chief investigator for this study was also the unit co-ordinator for IM560.) The research officer read the relevant portions of all exam scripts three times. Following the first reading, a preliminary indicator and set of categories was identified for each self-report measure in the evaluation framework. During the second reading, participants' scripts were classified by category and typical statements for participants in each category recorded. After this first round of coding, the unit co-ordinator reviewed indicators, categories, sample texts, and the research officer's detailed descriptions of how the indicators and categories were identified. Some minor revisions were made and all texts reviewed again. This approach achieved both consistency in investigators' understanding of the codes and texts and consistency in coding among the texts themselves.

As a result of the textual analysis, we were able to identify whether or not students related their learning to their participation in the GVT, as distinct from other course activities. We were also able to measure student satisfaction with the GVT along two dimensions: the extent to which the majority of participants found working as a team member a positive or negative experience, and whether or not (regardless of their experience in the team) each participant was satisfied with the overall experience of learning through the GVT Collaboration.

Table 2. Examination questions on which textual analysis of student satisfaction and learning through participation was based

Satisfaction with team experience

Team experience was coded either mostly negative (e.g., "we did not [establish team processes] and it led to many such messages asking if life did in fact exist within our group, which wasted lots of time and caused plenty of frustration") or mostly positive. Participants coded as having a mostly positive experience included those who made only positive remarks about relationships within the team and those whose reports indicated that while problems may have occurred within the team, the problems were resolved.

Overall satisfaction with the GVT Collaboration as a learning experience

Satisfaction with the process of learning was coded low or high. A value of high was assigned when participants' comments indicated pleasure in identifying behaviours or processes that resolved difficulties encountered in their teams, and to those participants who specifically expressed pleasure or enjoyment at having had an opportunity to participate in the project. A value of low was assigned to participants who reported their experience in a way that suggested that, although they may have learnt from the exercise, the learning experience was not one they enjoyed.

Past participants' comments on their experiences with the GVT suggested that satisfaction may be associated with the Internet skills of the participants. Satisfaction may also vary with team virtuality. We therefore explored the relationship between satisfaction and Internet skill scores and team virtuality.

Additional student satisfaction data were available through independent evaluation of the course and from unsolicited email sent to the course co-ordinator during and after the course. UWA conducts independent evaluations of courses using a system known as SPOT (Student Perception of Teaching <http://www.csd.uwa.edu.au/spot/>). The SPOT evaluation for IM560 was conducted by written survey in the classroom, in the course co-ordinator's absence, 2 weeks prior to the end of the GVT . Student responses were processed by a central university group and summaries returned to the lecturer with all student identifying data removed. While SPOT evaluations refer to the course as a whole, responses to open-ended questions about the strengths and weaknesses of the course provided some insight into reasons for student satisfaction with the GVT project. These reasons are discussed with the results of the study.

The relationship between learning and participation

We used two methods to study whether learning was associated with GVT participation: analysis of student reports and statistical analysis.

Student reported relationship between participation and learning

We drew students' reports of the relationship between participation and learning from textual analysis of exam scripts. For example, students were asked to "explain why team members should follow the guideline[s for effective global teamwork]... Give supporting evidence from your reading or your experience ... during this trimester." (See Table 1, question 1c) for complete question.) Responses that included examples drawn from participation in the GVT Collaboration demonstrated that learning resulted from participation in the exercise.

Participant reports of learning through participation in the GVT was classified on three levels. The first level (no explicit relationship) was assigned to those participants who presented no external support for their learning or who related their learning to course activities other than the GVT Collaboration. The second level (participation and behaviour) was assigned if a student related experience as a participant in the GVT Collaboration to their learning about team behaviour and performance. The third level (participation, behaviour and management) was assigned to participants who related their experiences in the GVT Collaboration not only to team behaviour and performance but also to broader business and management issues.

Statistical analysis of relationship between participation and learning

We used statistical techniques to explore further the relationship between participation and learning outcomes. Each of the learning outcome measures, project score and topic score, was used as a dependent variable in two separate analyses.

Two measures of the independent variable, participation, were used in each analysis: quality of individual participation and quality of team experience. Quality of individual participation was represented by peer evaluations. Peer evaluations were provided by the GVT Collaboration administrators at the University of Texas. Each team member was asked to evaluate each of the other members of the team on the 5 point scale described in Table 1. Each participant's evaluation score was the arithmetic average of the peer evaluation scores provided fellow team members. Quality of team work was represented by student reported satisfaction with the team experience.

Both project score and team experience may have been affected by differences in team composition. We therefore controlled for team virtuality (wholly virtual or partly virtual) in the analyses. Similarly, both project scores and topic scores are likely to reflect ability to perform as an MBA student. We controlled for the effect of individual ability through inclusion in the analysis of weighted average performance in the MBA prior to commencing the semester in which the GVT was run. We also controlled for the effect of prior Internet skills, in case lack of facility with the Internet created a barrier to participation or learning or both.

Results

This section describes learning outcomes and student satisfaction before examining the relationship between participation and learning. Correlations between all variables are in Table 3.

Table 3 Correlations among variables

Learning outcomes

All UWA students who participated in the GVT Collaboration were members of a team that produced a business plan which scored a pass grade. All earned more than 50% on exam questions that addressed the topics covered by the project, and there were no significant differences among learning scores on each of the three topics. Participant scores on project and topic learning score are recorded in Table 4. The scores were not correlated (r = .19, n = 42, p = .23).

Table 4. Student scores

Topic scores (mean = 74.9, sd = 9.8) were similar to those obtained to date in their MBA (mean = 74.1, sd = 5.4). Project scores were significantly different (paired t-tests with df = 40; t= 2.145, p = .038), but the mean (71.2) was within the range of mean scores (67 to 74) that the School expects students to attain in core units. The learning scores obtained from the GVT Collaboration were therefore not greater than the scores students would be expected to gain through other methods of learning about these topics.

Student satisfaction

Of the 28 participants who described their team experience in evaluative terms, equal numbers reported having a mostly positive or mostly negative team experience. Satisfaction with team experience did not vary with Internet skills or team virtuality.

Most students were satisfied with the process of learning through the GVT project, as shown in Table 5. Of the 39 students whose satisfaction with learning through the exercise could be coded, 30 (76.9%) volunteered that they had enjoyed the process or made other statements indicating that they were pleased with this approach to learning. On the other hand, nearly one quarter (9, 23.1%) were not satisfied with the GVT Collaboration as a method of learning. Overall satisfaction did not vary with Internet skills or team virtuality.

Table 5. Student satisfaction with learning

All participants who had a positive team experience (14) reported high satisfaction with learning through the GVT Collaboration. Half of those who had a negative team experience (7) also reported high satisfaction with learning through the GVT Collaboration, while 5 other participants in this group did not find the international collaborative approach particularly satisfactory. Satisfaction with learning appears to be independent of team experience (continuity corrected chi-square = 2.35, df = 1, p = .13).

Students' responses to the independent SPOT evaluation show high satisfaction with the course as a whole. Responses to relevant closed response items are summarised in Table 6. The practical work in the course was well received, and most students felt there was a good balance between theory and practice. Although more than three-quarters of the students were satisfied with the amount of work required, access to equipment, and forms of assessment, satisfaction with these aspects of the course was lower than satisfaction with content and learning.

Table 6. Student satisfaction with the course as a whole: Responses to items affected by the GVT.

Responses to the two open-ended SPOT questions, "What aspects [of the course] do you feel are the best?" and "Do you have any suggestions for change?" provided a little more insight into the role of the GVT in the course. Of the 20 positive remarks made about the course, 2 referred directly to the GVT and 5 indirectly to course elements which included the GVT; these remarks all affirmed the observations made on the closed response items. In addition, one student found the "GVT interesting but ..." added, "... topic for GVT assignment too difficult -- required too much prior knowledge and work load far more than estimated 20-30 hours." This was the only one of the 5 negative comments about the course to refer either directly or indirectly to the GVT, and it accorded with the course co-ordinator's observation that many students were putting in many more hours than the GVT project designers had anticipated. Overall, response to the SPOT survey suggest that some improvement in satisfaction could be obtained by reducing the overall work-load for the unit, providing students with greater opportunity to vary the proportion of their final grade attributed to the GVT project, and providing better access to faster computers and communication tools such as ICQ for students who rely on on-campus computer laboratories.

The relationship between participation and learning

The relationship between participation and learning was measured directly from students' reports, and explored in statistical analyses. In this section, the nature of the students' participation will be described before the relationships between learning from participation.

Participation

Peer evaluations were available for 39 of the 42 participants in this study. The distribution of peer evaluation scores is shown in Table 7. Eighteen of the participants (46.2% of those with peer evaluations) were given the highest possible score, 5 (excellent) by all members of their teams. The mean peer evaluation score was 4.62, with a standard deviation of .4. Only one UWA student received a peer evaluation rating below "average". The students in this study were among the highest rated participants in the GVT for participation.

Table 7. Quality of performance as a team member: Mean peer evaluations (study participants)

Student reported relationship between participation and learning

The majority of participants explicitly related their experiences as members of virtual teams to their learning. Table 8 shows that 81% (34) were able to relate their experiences as participants to their learning about virtual teamwork. More than half of these students (18, 42.9% of the total) were able to relate their experiences not only to their own behaviour as members of virtual teams but also to more general organisational and business implications of dispersed computer-supported teamwork.

Table 8. Student perception of learning

The level of reported learning through participation and satisfaction with learning in this way were related (chi-square = 11.479, df = 4, p = .022). Higher levels of satisfaction were reported by more students who related their learning to their participation in the GVT than by those who did not explicitly associate their learning with participation.

Participation and Project score

Inspection of the relationships between peer evaluations, team experience, and project scores showed that the distribution of project scores against peer evaluations reflected two distinct groups of participants: those with uniformly excellent peer evaluations and those with evaluations that included at least one rating below excellent. Figure 1 shows differences in project scores by peer evaluation group and team experience.

The figure shows that participants rated excellent by all their peers were members of groups that received lower average project scores overall. Project scores were particularly low for those excellent-rated participants who reported having had a negative team experience. All other participants received similar project scores, regardless of their team experience. From informal discussions with students during several GVT collaborations, we know that some students take over production of the group business plan if they perceive their groups are not performing and complete the plan alone or with one other member of the group. One student in this category e-mailed the course co-ordinator: "As I completed the plan pretty much by myself ... I guess a pass is OK". When a student takes project completion out of the hands of a team, the resulting output is likely to be of lower quality.

One-way analysis of variance confirmed that the effect of peer evaluation was significant (model F = 3.962, df = 3,12, p = .022) while the effect of team experience was not. None of the co-variates (team virtuality, Internet skills or student ability) affected these relationships. (There were insufficient numbers in all cells to complete a reliable two-way analysis of variance.)

Participation and Topic learning

Instructor-assessed learning about project goals, as measured by topic learning score, was not correlated with participation, as shown in Table 3. It was correlated only with prior Internet skills and ability.

To estimate the effect of Internet skills and ability on topic learning, taking participation into account, we used multiple regression. The independent variables were Internet skills and ability as an MBA student. Peer evaluation and team experience were entered into the equation as dummy variables. Only ability was associated with instructor-assessed learning about project goals. Residuals plots showed that the four factor model tended to over-estimate the topic learning score, so a second regression was run with ability the only independent variable. The residuals plots were acceptable for this model which explained 29.7% of the variation in topic learning score (F = 17.94, p < .001). One percentage point of MBA average (ability) score was associated with .74 points of topic learning score (t = 4.24, p < .01).

Discussion

The results of this study have implications for future computer-supported international collaborations, integration of such collaborations into future courses, and methods of evaluation of educational innovations in the field. The results are summarised here before discussion of each of the implications.

Summary of Results

International collaboration through the GVT was successful in many ways. Students were able to produce the required business plan through computer-supported collaboration in internationally dispersed teams. Through the GVT, and other supporting course activities, they learnt enough to achieve examination scores that mirrored their achievement in other MBA units. Most students were able to explicitly relate their participation in the GVT Collaboration to their learning about the topics that were the goals of the collaborative exercise.

Most students reported that they found this approach to learning enjoyable or satisfying. Although a third of students felt that their teams had not functioned satisfactorily, negative team experience did not necessarily translate into low satisfaction with the GVT Collaboration as a learning activity. Students who associated their learning with their participation were the most satisfied with the GVT.

Despite these positive outcomes, there is room for improvement. One fifth of participants reported low satisfaction with the collaborative exercise; all of these students were critical of the way their team had functioned. Negative team experience was also associated with low project score for many students who took on a workload well above team average as measured by peer evaluations.

International Collaborations

Efforts to improve team functioning are likely to improve both student satisfaction and team outcomes from collaboration. Inclusion of team building activities of the kind described by Knoll and Jarvenpaa (1998) would improve team functioning. These activities were not included in the 1997 GVT Collaboration, but several teams independently adopted them. "Share as much personal information as you can to enable fellow team members to 'build a person' out of your electronic communication. This provides team members with some emotional attachment to the team project" advised a member of one of these teams. Another team member noted, "Regular correspondence was necessary to maintain momentum on the project and team spirit. We applied a rule of responding within one day to messages." Our experience in this and other international collaborations (Bielli et. al, 1999) confirms that time spent on team building and rule setting exercises much improves quality of student experience, quality of team output and student satisfaction.

Participation in the GVT was expected to result in learning about how to work in an internationally dispersed team, and 81% of participants drew a link between participation with learning these skills. Participation was also expected to result in learning about more general organisational and business implications of dispersed computer-supported teamwork. Although successful in achieving this goal, fewer students (42.9%) made a direct link between participation and learning about these broader issues. This demonstrates that it is possible to use a computer-supported international collaboration as an exercise for learning about issues that go beyond simply the skills needed to work as a member of a virtual team, but other techniques may be needed by some students.

It is likely that the desired learning may be achieved with greater student satisfaction if the collaboration were based on a less complex project than used in the 1997 GVT Collaboration. Few participating students had experience with business planning; they therefore had to learn about business planning at the same time as learning to work as members of a virtual team and learning about integrated software packages and the functioning of a virtual organization. It is, however, difficult to design a realistic and meaningful project. The authors have been involved in computer-supported collaborative projects which have involved joint case study analysis, staged work in which students in one country complete hand-off work completed in one stage to students in another country, and shared tasks in which different team members complete different tasks which are combined in a single final report. Each of these techniques has proved complex because the students have had to learn not only about the subject matter of the task and how to behave in a virtual team, but also how to structure and co-ordinate a project of the specified kind. It may be that we should set limited learning goals at this early stage in international collaboration while students are still developing basic collaboration and co-ordination skills.

Despite the reservations discussed above, the GVT Collaboration provided a satisfactory experiential learning exercise in information management. The project itself was well-designed and managed by an experienced team at the University of Texas, using simple and (mostly) reliable technology. We believe, though, that success required more than sound technology and administration.

Laurillard (1993) argued persuasively that computer-based technologies are most effectively used in higher education to provide students with opportunities to practice their learning from theory, that is to apply theory to practice. Nonetheless, students learn in different ways. Some students are better able to draw theory from practice once they have experienced the practice while others use practice to test and extend their understanding of the theory. The GVT Collaboration was incorporated in the UWA information management course in such a way that theory and practice were intertwined: theory through class meetings and readings, and practice through parallel participation in the computer-supported collaborative learning exercise. Students were prepared for the GVT exercise with introductory exercises which developed essential technical skills and with classes, guest lectures, readings, and a class discussion which drew attention to the key theories and issues in the field. They were encouraged to pay close attention to building their group into a well socialised team. This approach was appreciated by the students, one of whom noted as a "best" feature of the course in the SPOT evaluation, "[The course co-ordinator] has put a great deal of thought into the blend of ... activities and the sequence in which ideas are introduced.".

Others have described less successful experiments with computer-supported collaborative learning in management education. These experiments emphasised the possibilities of the technology more than the human aspects of using the technology effectively to enable effective learning (Cramton, 1997; Guice, 1997). With the benefit of several years' experience, we are learning to apply what Guice elegantly describes as the reciprocal nature of the social and technical aspects of course design. It is unrealistic and unfair to expect students to learn new behaviours, form new relationships, and learn about new topics and methods in a short period of time, with the pressure of assessment and without guidance on social aspects of technology use, particularly when the technologies and the networks on which we rely for collaboration do not always function optimally. Technical limitations can be overcome with social design, and course designers should take care to help students learn as much as possible about how attention to the social aspects of virtual collaboration can overcome technical limitations as part of their preparation for participation in collaborative exercises.

The most powerful way to evaluate an educational innovation is to compare the outcomes for learners who have participated in the innovation with outcomes for those who have learnt about the specific knowledge and skills in other ways. This approach to evaluation is, however, seldom available to educators in the field. The approach taken in this study was effective. It provided several perspectives on the exercise: instructor-assessed achievement of learning goals, peer evaluations of student participation in teams, student perception of learning through the computer-supported collaborative learning exercise, independent evaluation of student responses to the course in which the exercise was offered, and statistically measured relationships between participation and learning outcomes. Student self-reports of satisfaction, and of the relationship between participation and learning, were particularly useful, enabling us to observe what learning did in fact arise from participation, and to learn about how students responded to this approach to learning.

Conclusion

This study showed us that students respond well to computer-supported collaboration as a technique for learning about behaviours in computer-supported international teams. We have not yet perfected the technique, nor do we know how widely it can be applied in management education and training. Nonetheless, incorporated appropriately into course design, computer-supported collaborative learning about virtual team work is effective.

Acknowledgments

This project was completed with the assistance of an Australian Research Council Small Grant administered through the University of Western Australia, a Teaching and Learning Development Grant from the University's Faculties of Economics & Commerce, Education and Law, and of Patrizia Zuppini, IMI-Fabi SRL, and colleagues at Bocconi University in Milan, Italy. Sirkka Jarvenpaa and Kathleen Knoll at the University of Austin, Texas, generously gave their permission to study aspects of the Global Virtual Team project which they had initiated and developed.

References

Alavi, M. (1994). Computer-mediated collaborative learning: An empirical evaluation. MIS Quarterly, 18, 159-174.

Alavi, M., Wheeler, B. C. & Valacich, J. S. (1995). Using IT to re-engineer business education: An exploratory investigation of collaborative telelearning. MIS Quarterly, 19, 293-312.

Allen, T. J., & Hauptman, O. (1990). The substitution of communication technologies for organisational structure in research and development. In J. Fulk & C. W. Steinfield (Eds.), Organizations and Communication Technology (pp. 275-294). Newbury Park, CA: Sage.

Angehrn, A., & Nabeth, T. (1997). Leveraging emerging technologies in management education: Research and experiences. European Management Journal, 15(3), 275-305.

Bielli, P., Renzi, S., Kumar, K. & Klobas, J. (1999). Collaborative work among students: Enrichment or frustration? An experience from CEMS schools. In P. Marquet, S. Mathey & E. Nissen (Eds) Proceedings of IN-TELE 98. Berlin: Peter Lang.

Cramton, C. D. (1997). Information problems in dispersed teams. In L. Dosier & J.B. Keys (Eds.) Academy of Management Best Papers Proceedings (pp. 298-302). Georgia: Southern University.

Davison, S. C. (1994). Creating a high performing international team. Journal of Management Development, 13, 81-90.

Gorton , I. & Motwani, S. (1996). Issues in co-operative software engineering using globally distributed teams. Information and Software Technology, 38, 647-655.

Guice, J. (1997). Rethinking the wired classroom. IPCT: Interpersonal Computing & Technology, 5, 15-27.

Hazemi, R., Hailes, S., & Wilbur, S. (Eds.). (1998). The Digital University: Reinventing the Academy. London: Springer.

Henry, J. E., & Hartzler, M. (1997). Tools for Virtual Teams: A Team Fitness Companion. Milwaukee, WI: ASQC Quality Press.

Ives, B., & Jarvenpaa, S. L. (1996). Will the Internet revolutionize business education and research? Sloan Management Review, 37(3), 33-41.

Jarvenpaa, S. L., & Ives, B. (1994). The global network organization of the future: Information management opportunities and challenges. Journal of MIS, 10(4), 25-57.

Jarvenpaa, S. L., Knoll, K., & Leidner, D. L. (1998). Is anybody out there? Antecedents of trust in global virtual teams. Journal of MIS, 14, 29-38.

Johnson, D. W., & Johnson, R. T. (1989). Cooperation and Competition: Theory and research. Edina, MN: Interaction Book Company.

Katzenbach, J. R., & Smith, D. K. (1993). The discipline of teams. Harvard Business Review, March-April, 111-120.

Keen, P. G. W. (1990). Telecommunications and organisational choice. In J. Fulk & C. W. Steinfield (Eds.), Organizations and Communication Technology (pp. 295-294). Newbury Park, CA: Sage.

Kimber, D. (1996). Collaborative learning in management education: Issues, benefits, problems and solutions. A literature review [WWW document (UltiBASE publication) http://ultibase.rmit.edu.au/Articles/june96/kimbe1.htm]. Melbourne: Royal Melbourne Institute of Technology.

Knoll, K. & Jarvenpaa, S. (1998). Working together in global virtual teams. In: M. Igbaria & M. Tan (Eds)The Virtual Workplace (pp. 2-23). Hershey, PA: Idea Group Publishing.

Laurillard, D. (1993). Rethinking University Teaching: A Framework for the Effective Use of Educational Technology. London: Routledge.

Leidner, D. E., & Jarvenpaa, S. L. (1995). The use of information technology to enhance management school education: A theoretical view. MIS Quarterly, 19(3), 265-291.

Nohria, N., & Eccles, R. (1992). Face-to-face: Making network organizations work. In N. Nohria & R. Eccles (Eds.), Networks and Organizations: Structure, Form, and Action (pp. 288-301). Boston, MA: Harvard Business School Press.

Sproull, L., & Kiesler, S. (1991). Connections: New Ways of Working in the Networked Organization. Cambridge, MA: MIT Press.

Appendix: Variables used to evaluate the global virtual team project