|
| [ ASCILITE ]
[ 2004 Proceedings Contents ] |
This paper outlines the context and theoretical basis for the design, implementation and evaluation of an online conference conducted in 2003. The purpose of the conference was to provide postgraduate distance learners with an opportunity to interact with human factors and healthcare professionals, thereby providing them with exposure to this emerging community of practice. The conference was delivered through a WebCT site and stimulated various modes of interaction. The paper discusses the design and format of the conference and details an analysis of the online transcript that shows development of learning communities as "comfort zones" within which the participants could communicate in a common language and atmosphere of understanding over the 2 days that the conference ran.
All postgraduate students in the Online Program study part time, most are mid-career professionals with well developed core disciplines such as software engineering, health care, work place safety, defence, aviation, and education. Students are well qualified in these primary disciplines, with 30% already holding postgraduate degrees. Study is seen as a means of enhancing vocational opportunities, acquiring up to date information, and as a way of differentiating themselves from non-qualified practitioners. However, for most students HF involves knowledge and practices that are different to those learnt in their core disciplines. In a HF context students are outside of their comfort zones. At present most students are located within Australia, however, students from China, Hong Kong, Singapore, the UK, USA and Canada are also enrolled. The average age of students is 35 and they have spent between 3-34 years in the workforce. Few students aspire to doctoral study or to making the transition to academic careers.
For most students postgraduate study is not a life priority and is often the first casualty when work and family pressures increase. As a consequence attrition rates can be high; ours were in the order of about 30%. Non-completing students claimed two main reasons for dropping out: a lack of perceived relevance of the program to practice; and, a sense of isolation in the online environment. In order to promote program relevance we needed to ensure that the program and course content could demonstrably facilitate students' entry into HF as a field of practice. A way to do this was to define and assess courses against a set of graduate attributes. The Online Conference was conceived as part of a broader strategy to deal with isolation.
We embraced GAs as a tool that would help to guide the development of an integrated program, provide a framework for decision making about course content, assessment and review, and which would help us to communicate student outcomes. The GAs also provided a basis for articulating the practices and values of our own discipline, particularly in light of Parker's (2002) belief that teaching and learning are part of the "ongoing and inclusive job of forming, reflecting on and establishing disciplinary understanding" (p.378). We maintained that graduates should be able to advocate and enable the effective design of human technology systems from a HF perspective.
The GA that most inspired this conference was:
Effective Communication - Graduates have the ability to communicate to academic and domain specific audiences, journals and professional publications, can communicate effectively for practice related purposes by writing stakeholder reports, discussion papers and strategic documents. Graduates can communicate across disciplines, using language and key concepts of relevance to multidisciplinary teams.
The intellectual substance of a community is supported by social interaction, (opportunities to connect with others), and shared experience including the exchange of meaningful and reflective dialogue among community members. Interaction, practice and shared experience bring coherence and social structure to an evolving community (Bielaczyc, K. & Collins, A., 1999; Wenger, E., 1998) as it is through practice that community members form relationships with each other and with their work. According to Wenger mutual engagement, shared repertoire, and joint enterprise are essential characteristics of cohesive communities. Thus a community may be defined and quantified by the network of connections made between individuals and by the content of their communication over time. Communities of practice form the comfort zone in which the practitioners in a particular field work, within familiar and commonly understood principles, approaches and values.
The importance of this perspective for university teaching is the highlighted need for curriculum and learning experiences that facilitate development of both substantive knowledge and practice oriented understanding through mutual engagement, shared repertoire and joint enterprise. Designing learning experiences for students must therefore involve strategies for exposing students to the language, discourses and modes of communication that the field favours, and providing opportunities for students to 'practice' using their knowledge with other members of the community (Huber, 2003; Parker, 2002). Thus we need to provide examples of disciplinary thinking and intellectual engagement in an authentic and explicit manner that facilitates student participation in the 'business' of the discipline. From this perspective the online environment may be an impoverished platform. Students rarely have the opportunity to observe other professionals engaged in academic and professional discourses.
In addition to providing a context for broad based exchange among students that might facilitate a sense of community, we also observed that if graduates are to effectively advocate for a HF perspective, they need skills that are best learnt through engagement with experienced professionals. The challenge for us was to provide opportunities for geographically distributed students to interact with a range of professionals in order to create a distributed community in which they could feel comfortable in an online environment.
The online conference was an attempt to address these needs. The objectives of the conference from this perspective were to explore available technologies in terms of their ability to support social interaction online; to design activities that would support different types of social interaction; and to develop measures and evaluation approaches that would allow us to track and chart the development of a learning community.
HF is a mature field in areas such as aviation. A conference based in this area would have attracted experts but may have excluded HF novices. Human factors in health care is a new field which has gained prominence with the release in 1999 of the United States Institute of Medicine's report To Err is Human. Health care also presents challenges for HF professionals. Health care systems are socio-technical-biological systems (Miller, 2004), they are highly open, and patients present themselves at any time with an array of problems, whereas other HF contexts are relatively closed and less subject to external perturbation. Thus health care challenges HF professionals to apply and discuss conventional approaches in a new domain with practitioners who are not familiar with HF language. Similarly, health care practitioners can provide rich descriptions of their work context, but they need to do so in accessible language. In Becher and Trowler's (2001) terms, the Online Conference would serve to bridge the gap between the health care and human factors disciplines with the aim of creating a new community through an exchange of language, knowledge and values in a way that could include students. The theme of the conference therefore became Human Factors in Medicine.
A non-interactive conference program was permanently available on this site and the interactive program was progressively revealed on the home page to control the times at which participants could access conference resources. The Opening, Closing and Keynote presentations were delivered as QuickTime movies. The QuickTime movies consisted of audio narration supported by slides with dot-points, quotes, graphs, diagrams and photographs rather than text alone: The combination of description and visual representation is understood to provide a more powerful learning experience (Laurillard, 1993). The presentations could be played as often as the participants wanted. This benefit outweighed the danger that the pre-recorded movies might have detracted from the sense of live participation.
| Date / Time | Session |
| 28 November | Conference registration |
| 29 November | |
| 0800-0830 | Opening Address Dean, School of Medicine, UQ. |
| 0830-1130 | Scenario 1: Community Health Presented by a General Practitioner from the UK. |
| 1130-1230 | Panel Session 1: What do you do when your corporate sponsor leaves? Assistant Professor, University of Maryland Medical Centre, Baltimore Risk and Safety Manager of the Clinical Governance Unit, Hunter Area Health Service. Senior Lecturer, Key Centre for Human Factors, UQ. |
| 1330-1400 | Keynote Presentation: Communication and Technology in Health Care. Co-director, Centre for Health Informatics, UNSW. |
| 1430-1700 | Scenario 2. Organisational Policy Presented by the Risk and Safety Manager of the Clinical Governance Unit, Hunter Area Health Service. |
| 30 November | |
| 0830-0930 | Panel Session 2: Options for Investigating HF Issues in Health Care Environments Professor of Human-Computer Interaction University of Middlesex, UK Senior Lecturer and Director of Research Dept of Anaesthesia, Monash Medical Centre, Melbourne Clinical Professor of Anaesthesiology and Intensive Care Hadassah Hebrew University Medical Centre. |
| 0930-1230 | Scenario 3. Anaesthesia and the Operating Room Presented by an Anaesthetist at the Monash Medical Centre. |
| 1230 | Closing Remarks Director, Centre for Online Health, UQ |
The scenario based discussions (see Table 2) were a major divergence from a conventional conference. Health care practitioners prepared three case based discussion scenarios. The first scenario was about the support of elderly patients living at home. The presenter of the second scenario was a senior patient safety officer in a regional health care network whose scenario (which informed the second discussion activity) focused on how corporate policy sets the stage for patient safety. The third scenario focused on risks to patients' safety in a specialised area of acute hospital health care. All of the scenarios were hypothetical, but based on real events.
At the beginning of each scenario activity participants read the case studies and viewed the supporting photographs. After 30 minutes, participants were invited to ask clarification questions of the presenters. These were important in establishing a common language and a shared understanding of the scenario context. Following the clarification questions, exploratory questions were presented. The purpose of these questions was to create conceptual links between the events represented in the scenarios and HF concepts.
The manner in which the different scenarios were constructed generated different forms of interaction. The second scenario presenter left significant information out. This provided an opportunity for participants to 'play detective', sleuthing through the events to uncover key aspects and in doing so discover a language within which they could express their ideas. The ill-defined nature of the scenarios was engaging and stimulated deep reflection. The scenario discussions were driven very much by practitioners and couched in the language of health care but understandable to lay people.
| Activity | Resource/s task/s | Participants' mode of interaction |
| Conference Registration | Login to the password protected WebCT website | Participants and presenters introduced themselves on the asynchoronous text based conferencing tool (WebCT's Dicussion tool). |
| Opening, Closing and Keynote Presentations | View QuickTime presentations (slides and audio narration) | Within a dedicated activity forum in the Discussion tool participants raised questions, discussed issues and contrasted them with their own experiences. Presenters clarified remarks and responded to questions. |
| Scenario Discussions | Read scenarios and view supporting photographs | Within separate activity forums in the Discussion tool participants asked clarification questions, presenters and participants discussed exploratory questions and a moderator summarised the discussion. |
| Panel Sessions | Observe HF experts engaged in a moderated discussion | Within a synchronous, text based conferencing platform (WebCT's Chat tool) panelists discussed a set topic for thirty minutes. Participants followed the discussion but did not participate until the end. |
Separate forums within the WebCT Discussion tool were used for each scenario activity. Within each forum the Discussion tool's 'thread' feature allowed parallel conversations to take place simultaneously. Multiple conversations emerged when moderators posed questions and HF professionals extended and illustrated issues using examples from their own experience. Participants were also free to present their own questions. One scenario generated 119 messages in the space of 2:5 hours and was organised by 11 threads with meaningful headings (i.e. the initial message's subject line). In this way threaded discussions became highly self organising and freed the presenter from the responsibility of answering every question. Because these discussions involved all participants, peer to peer learning was facilitated as opposed to a vertical transfer of information from expert to learner (Wegner 2003).
This variety of communication modes and tools facilitated several types of social interaction. The scenarios allowed free flowing discussion among delegates, often across multiple emerging topics. Discussion was lively and appeared to become more reflective as the session progressed. At the end of the conference participants spontaneously thanked and farewelled each other. This message from a participant suggests she felt comfortable as evidenced by her familiar message.
Susanne from Denmark is still awake - but I will go to bed now - thank you for an interesting conference.
The keynote address, delivered by a speaker who is both a human factors professional and a medical practitioner, introduced language and perspectives common to all delegates. Themes presented in the keynote were referred to and elaborated upon in subsequent discussion. The immediacy of the chat tool, with which the panel sessions were conducted, allowed delegates the opportunity to passively observe the language and interactive behaviour of panellists as students might observe the behaviour of academic colleagues.
Thus we achieved our first two objectives: to establish a technological infrastructure that would support interaction and to design activities that were socially engaging. Our third objective was to develop an evaluation approach that might be used to quantify a learning community.
Testing these hypotheses required the detailed mapping of the message flows (interpersonal interconnections) and message content (concepts). This was done with the aid of a sophisticated social network and concept mapping tool. Leximancer [http://www.leximancer.com/] is a software package that supports the analysis of natural language. Using statistical algorithms, Leximancer extracts semantic and social network maps from text data, such as transcripts. Manually produced concept maps have been used extensively in education and can assist with comprehension of large bodies of text but are limited given the logistical difficulties of manual sorting and grouping. The Leximancer technologies are based on computational linguistics, machine learning, and complex systems. The result is a form of automatic relational content analysis.
The following concept maps created using Leximancer are based on the interactions between delegates and the content of their discussion during the scenarios. We chose to analyse discussion generated by the scenario activities (rather than other conference activities) because in these activities delegates were asked probing questions that stimulated conversation over extended time frames (2.5 hours for each scenario). Also, unlike the one off presentations, three scenarios were presented during the conference, thereby allowing a comparative analysis of concept maps over time.
Figure 1: Scenario 1 (Saturday am)
The scale of the maps presented above make them difficult to read however at full scale they show the progressive interconnection of both delegates and concepts across the conference. In the map for Scenario 1, concepts and delegates are widely distributed across the concept space. There are few and certainly not strong connections between people and between concepts. The conference facilitators feature strongly as interaction and conceptual linchpins. In Scenario 2, the connections (interactions) between delegates contract towards the centre of the concept map although there are still some peripheral delegates and certainly there are peripheral or unconnected concepts. The facilitator's role is less prominent. By Scenario 3, the majority of delegates are strongly connected towards the centre of the map and concepts are highly interconnected. The conference facilitator's input is outweighed by the input of others and is limited to procedural involvement only.
Figure 2: Scenario 2 (Saturday pm)
Figure 3: Scenario 3 (Sunday am)
Although no formal survey was conducted, participants' voluntary feedback about the conference confirmed the findings shown in the Leximancer evaluation. Delegates reported on enjoying both the formal presentations as well as the ensuing discussion and their feedback indicates a sense of connection with each other:
I enjoyed this presentation a lot and was very interested in the comments about healthcare systems being 'people, tools and conversations'.It has been a pleasure to participate in this discussion.
Thank you for the discussions today, which have given me food for thought. I am looking forward to tomorrow's meeting,
We were quickly able to move away from the specifics of the case to a more interesting and more general discussion,
In addition, the usefulness of Leximancer in evaluating Online Conferences has raised other possibilities for its use within the Online Program. Within each course, students' participation in weekly discussion topics are graded according to assessment criteria that include the quantity of messages contributed and the quality of messages (that is, the accuracy of information in the message; the relevance of the message; the inclusion of experiences and illustrations; evidence of reflection and relation back to course materials and other students' postings). The task of assessing course discussion is time consuming and potentially subjective and therefore unreliable. Leximancer may be helpful in supporting assessors in providing greater objectivity in the analysis of student contribution to online discussions and may also be used to assess the authenticity of student assignments. Throughout the Program we aim to provide an integrated learning environment across students' courses of study. Again Leximancer may prove to be valuable in assessing the linkages and elaboration of concepts across the formal course content within the Program.
One of the most exciting aspects of the conference and its format was the self organising nature of discussion within a loose framework. Unlike most conventional conferences where the subjects under consideration are set in advance, the topics in this conference emerged through the online interactions among participants. The themes and the content associated with these very much represent contemporary issues and have subsequently provided rich material for the development of student course work especially the scenarios and examples given throughout.
We intend to adopt aspects of the Online Conference for use in the Online Program. Using the WebCT Discussion tool as a forum for shorter seminars that involve guest speakers will provide opportunities to assess whether long term connections between students and speakers are developed. The conference succeeded in developing, at least on a temporary basis, an online community which provided a comfort zone in which students and practitioners could share their ideas and experiences. The challenge now is to extend the community into the whole online program as part of students' progression into the HF field.
Becher, T. & Trowler, P. (2001). Academic tribes and territories (2nd Ed). Buckingham, UK: The Society for Research into Higher Education and Open University Press.
Billett, S. (2001). Knowing in practice: Re-conceptualising vocational expertise. Learning and Instruction, 11, 431-452.
Boyer, E. L. (1990). Scholarship revisited. Princeton, NJ: Carnegie Foundation for the Advancement of Teaching.
Delanty, G. (2001). Challenging knowledge: The university in the knowledge society. Buckingham, UK: The Society for Research into Higher Education and Open University Press.
Gibbons, M., Limoges, C., Nowotny, H., Schwartzman, S., Scott, P. & Trow, M. (1994). The new production of knowledge. London: Sage.
Giddens, A. (1991). Modernity and self-identity. Cambridge: Polity Press.
Gosling, D. (2003). Philosophical approaches to academic development. In H. Eggins & R. Macdonald (Eds), The scholarship of academic development. Buckingham, UK: The Society for Research into Higher Education.
Healey, M. (2000). Developing the scholarship of teaching in higher education: A discipline-based approach. Higher Education Research & Development, 19(2), 169-189.
Huber, M. T. (2003). Disciplines and the development of a scholarship of teaching and learning. In R. Blackwell and P. Blackmore (Eds), Towards strategic staff development in higher education. Buckingham, UK: The Society for Research into Higher Education and Open University Press.
Hung, D.W.L. & Chen, V.D.T. (2002). Learning within the context of communities of practices: A re-conceptualization of tools, rules and roles of the activity system. 2002 International Council for Education Media. London: Routledge.
Laurillard, D. (2002). Rethinking university teaching: A conversational framework for the effective use of learning technologies (2nd ed.). London: Routledge Falmer.
Light, G. & Cox, R. (2001). Learning and teaching in higher education. London: Paul Chapman Publishing.
Lueddeke, G. R. (2003). Professionalising teaching practice in higher education: A study of disciplinary variation and 'teaching-scholarship. Studies in Higher Education, 28(2), 213-228.
Miller, A. (2004). Developing an integrated information representation for medical intensive care unit patients: Generalising work domain analysis. Unpublished PhD thesis. School of Psychology: University of Queensland, Australia.
Neumann, R., Parry, S., & Becher, T. (2002). Teaching and learning in their disciplinary contexts: A conceptual analysis. Studies in Higher Education, 27(4), 405-417.
Parker, J. A. (2002). New disciplinarity: Communities of knowledge, learning and practice. Teaching in Higher Education, 7(4), 373-386.
Rice, E. R. (1995). Foreword. In R. M. Diamond & B. A. Adam (Eds), The disciplines speak: Rewarding the scholarly, professional, and creative work of faculty. Washington, DC: American Association for Higher Education.
Smith, A. E. (2000a). Machine learning of well defined thesaurus concepts. Proceedings of the International Workshop on text and web mining (PRICAI 2000). Melbourne, Australia.
Smith, A. E. (2000b). Machine mappings of document collections: The Leximancer system. Proceedings of the Fifth Australasian Document Computing Symposium, Sunshine Coast, Australia. DSTC. [verified 2 Oct 2004] http://www.leximancer.com/documents/adcs2000.pdf
Watson, M. & Horberry, T. (2003). The value of an online, distance education program in human factors. Proceedings of the Ergonomics Society 2003 Annual Conference. Heriot-Watt University, Edinburgh, UK, 15-17 April.
Wenger, E. (1998). Communities of practice: Learning, meaning and identity. Cambridge, UK: Cambridge University Press.
Teaching and Learning Committee (2002). Report of the working party: Graduate Attributes and Curriculum Review. Brisbane, Australia: University of Queensland. [verified 2 Oct 2004] http://www.uq.edu.au/academic-board/policy/Curriculum_Review.pdf
| Authors: Anne Miller, Key Centre for Human Factors & Applied Cognitive Psychology, The University of Queensland, Brisbane Qld 4072 Australia. a.miller@humanfactors.uq.edu.au Mia O'Brien, Teaching and Education Development Unit, The University of Queensland, Brisbane Qld 4072 Australia. mia.obrien@uq.edu.au Greta Kelly, Teaching and Education Development Unit, The University of Queensland, Brisbane Qld 4072 Australia. greta@uq.edu.au Alethea Blackler, Key Centre for Human Factors & Applied Cognitive Psychology, The University of Queensland, Brisbane Qld 4072 Australia. thea@humanfactors.uq.edu.au Please cite as: Miller, A., O'Brien, M., Kelly, G. & Blackler, A. (2004). Online conference case study: Creating a comfort zone within an emerging community of practice. In R. Atkinson, C. McBeath, D. Jonas-Dwyer & R. Phillips (Eds), Beyond the comfort zone: Proceedings of the 21st ASCILITE Conference (pp. 663-672). Perth, 5-8 December. http://www.ascilite.org.au/conferences/perth04/procs/miller.html |
© 2004 Anne Miller, Mia O'Brien, Greta Kelly & Alethea Blackler
The authors assign to ASCILITE and educational non-profit institutions a non-exclusive licence to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. The authors also grant a non-exclusive licence to ASCILITE to publish this document on the ASCILITE web site (including any mirror or archival sites that may be developed) and in printed form within the ASCILITE 2004 Conference Proceedings. Any other usage is prohibited without the express permission of the authors.