Journal of Instructional Science and Technology ISSN: 1324-0781 Editors-in-Chief: Olugbemiro JEGEDE (jegede@ouhk.edu.hk) and Som NAIDU(s.naidu@meu.unimelb.edu.au)

---

Volume 1 No 4, November 1996
- - - Article 2 - - -

An Evaluation of the Queensland Open Learning Network Audiographic Conferencing Professional Development Programs

by

Olugbemiro J JEGEDE
The University of Southern Queensland, Toowoomba, Australia

Anne GOOLEY & Stephen TOWERS
Queensland Open Learning Network, Brisbane, Australia

[ Abstract | Introduction| Methodology | Results andDiscussion | Summary and Conclusion | References ]

Abstract

The literature indicated that although there was evidence that audiographicsconferencing was used for education and training purposes, relatively little informationon professional development programs was available. The objectives of this study were to(1) evaluate participants' perceived value of the audiographics professional developmentprogram, (2) identify areas for revision and improvement, and (3) collect and disseminateinformation about professional development programs for audiographics conferencing of theQueensland Open Learning nertwork (QOLN). A questionnaire was administered to participantsof an audiographic professional development program to gather data for a comprehensiveevaluation of the training program. The data was quantitatively and qualitativelyanalysed. The results of the evaluation indicated, amongst others, that (i) the majorityof the participants found the training workshop very beneficial, easier than they hadexpected, and that computing skills had no relationship with their mastery of the use ofaudiographic conferencing equipment; (ii) they were pleased with the interactivitycharacteristic of the audiographic equipment, and the learning environment requirementsfor the use of audiographic conferencing; and (iii) participants considered audiographicconferencing. Based on the results, a number of recommendations were made relating topolicy on training and professional development opportunities for teachers usingaudiographics conferencing, encouraging teaching staff and instructors to useaudiographics conferencing, and the need to disabuse the minds of people of the myth thathigh computer literacy is very essential for successful audiographic conferencing.

Introduction

In 1993 the Queensland Open Learning Network installed audiographic conferencingfacilities in each of its 40 Open Learning Centres (OLC's) to provide a human interfaceand technology infrastructure for the delivery and reception of learning programsthroughout Queensland. Over this period, demand has emerged for personnel to be trained touse audiographic conferencing education and training activities. Apart fromfamiliarisation of the technical aspects of audiographics conferencing there was a lack ofprofessional development programs to train teachers, lecturers and other facilitators howto use the technology appropriately. The aims of the professional development program wereto:

1. design, develop, deliver, and evaluate an audiographic-teleconferencing skills training program for a target group of lecturers.
2. pilot and compare open learning and face-to-face delivery methods for the audiographic-teleconferencing skills training program with lecturers from two universities.
3. Refine and produce master copies of training materials.
4. Distribute details of the programs to higher education institutions providing distance education and open learning throughout Australia.

Computer mediated communications have become common in professional development withinthe past decade especially as their use in distance education and open learning becamevery pervasive. Such applications have included: continuing education for Francophone andAnglophone addictions workers across Canada ( Burge, 1993); the development of computerconferencing to support a face to face conference in Bangkok (Anderson & Mason, 1993);the Satellite On-line Searching Interactive Conferencing Experiment at the University ofPlymouth, United Kingdom (Hughes & Priestley, 1992); for access in remote areas(Robson, 1991); in teacher education (Westerman, 1990); a research-based communicationmodel of professional development (Wilsman, 1988); and, online professional development ininstructional design (de Vries, Naidu, Jegede & Collis, 1995). A summary of theexisting information on the use of computer mediated communications in professionaldevelopment indicated that:

• CMC are effective in ensuring equal access to group provision
• CMC benefits those who are disadvantaged by distance and social/economic factors
• CMC help both new and experienced educators develop as reflective practitioners
• CMC are cost effective, and involve real time linking of local groups of professionals with their peers/colleagues and with external expert colleagues
• CMC fostered positive response to the course and the technology as an effective way to learn material otherwise unavailable to them

However, a strong indication was found in the literature (see Johnston, 1992) to havemore information and training on CMC. If this could be demanded of the less difficulttypes of CMC (computer and audioconferencing), the need for training regarding theaudiographic conferencing could be well imagined. Audiographics conferencing is thetransmission of images and text between computers. Audiographics is a software packagethat runs on a personal computer and usually uses two standard telephone lines - one foraudio and one for data. It is used in conjunction with audio teleconferencing so thatparticipants may hear each other and also view the same information, diagrams or pictureson the computer monitors. While many sites can be linked together, 3-5 sites are common.Audiographics conferencing is often described under the rubric of 'teleconferencing' orsometimes the term 'computer mediated communication'. Audiographic facilities enhanceaudio conferencing by providing the means to display and share visual images. Audiographicconferencing provides lecturers/teachers and learners with the means to interactivelyshare graphic and auditory information across multiple locations. The shared workspaceenables participants from each site to contribute visual as well as audio information.This type of conferencing offers an alternative for the design, development and deliveryof education and training programs to remote and decentralised regions of Australia.Programs can be quickly and simply prepared and delivered to multiple locationssimultaneously. A geographically distributed target group can be aggregated to increasethe cost effectiveness of information dissemination and training cost effective. Animportant feature of the system is that education and training activities may originatefrom any location which enables and empowers small and large communities to deliver aswell as receive courses.

Audiographics can replace face to face instruction or print-based material as theprimary delivery strategy or can be used within a wide range of flexible delivery optionsto supplement or enhance learning programs (Gooley & Towers, 1996). A typical exampleis that courses can be delivered to a large audience via satellite broadcast (one-wayvideo, two way audio) followed by a series of small group audiographic conferencing. Muchthe same way that lectures providing mass information are followed by tutorials toencourage deeper learning and peer interaction.

Although there is evidence that audiographic conferencing has been used in a number ofprojects for delivery of instruction (Clark, 1989; Murphy, 1988; Ehrmann, 1990; Gilcher& Johnstone, 1988; Martin, 1986), there was nothing to indicate deliberate effort tomap out and evaluate its use for training. Especially those who would use the technologyon a regular basis to achieve distance education and open learning related objectives.Indeed, Blinn (1989) recommended the evaluation of this technology especially where longterm training requirements are conducted on a frequent basis to multiple remote sites.This underlies the objectives behind this project undertaken on the QOLN Audiographicconferencing professional development program.

The objectives of this evaluation were to: evaluate participants' perceived value ofthe audiographics professional development program; identify areas for revision andimprovement; and collect and disseminate information about professional developmentprograms for audiographics conferencing. This evaluation significantly departs from thetraditional approach by focussing on the training required to use such technologies foreducational applications.

Methodology

Sample

The sample included lecturers/teachers from tertiary education providers whoparticipated in a three day training workshop on audiographics conferencing. Allparticipants of the workshop were invited on a voluntary basis to complete thequestionnaire. A total of 30 questionnaires were distributed. 26 questionnaires werereturned.

Instrumentation

A questionnaire, The Audiographic Conference Training Program EvaluationQuestionnaire (ACTPEQ) was designed for participants undertaking the training programto gather data for a comprehensive evaluation of their training program. The questionnairecomprised three major sections. Section A was for demographic information which includeditems seeking pertinent information about the participants, that is; keyboarding skills,computing knowledge, status, and course level they currently teach at. Section B containeda total of 55 items in eight factors and sought responses on areas such as expectations ofaudiographic conferencing prior to, during and after the training program; interactivity;difficulty; learning environment; and education/cognitive values. Responses to items inthis section were based on a six point Likert types scale of “Strongly Agree” =5, “Agree” = 4, Neutral/Not Sure” =3, “Disagree” =2,“Strongly Disagree” =1, and “Does Not Apply” = 0. Section C was in afree response format allowing for comments on: easy and difficult features associated withaudiographic conferencing for education and training purposes; outcomes of the workshop;and suggestions for improving future workshops.

The ACTPEQ was developed and refined over two years and underwent several stages ofvalidation and trials. Sections of the questionnaire had been trialed in earlier studies(Jegede & Gooley, 1994; Gooley &, Jegede 1995) and the several drafts of theinstrument were reviewed by selected academics from two independent universities, andexperts in distance education and open learning, computer mediated communications, socialsciences research methodology, and biometrics. The results of the validation andreliability analyses indicated that the instrument exhibited a test-retest and split-halfreliability coefficients of 0.92 and 0.94 respectively. The alpha reliability for theitems in Section B ranged from 0.88 to 0.97. The major factors within the final instrumentcorrelated significantly well (p< .05) with coefficients ranging from 0.22 to 0.53.

Administration of the Instrument

The ACTPEQ was administered to participants who volunteered to evaluate their training,at the commencement of the training workshop. They were advised to complete it and toreturn it either to the facilitator or the address provided (as discretion of therespondent) once the workshop was completed. The 26 completed copies of the questionnairesreturned were collated ready for data analysis.

Analysis of data

The data collected were quantitatively and qualitatively analysed using the SPSS forWindows and the NU.DIST software respectively. For the quantitative analysis, descriptivestatistics, correlation coefficients, paired t-tests, and analysis of variance (ANOVA)were employed. The free response comments on aspects of the training workshop related to:easy features of audiographics; difficult features of audiographics, what was gained fromthe workshop; and, suggestions for improving future workshops. These were subjected todetailed exploration using the NU.DIST software in order to tease out salient factors anddetermine any trends in the reactions of the participants to the resources, content andorganisation of the workshop.

The 26 respondents comprised 16 males and 9 females. The majority of them 11 (44%) fellwithin the 36-45 years age bracket, 8 (32%) rated their keyboard skills as good orexcellent, 12 (48%) of them rated their computer knowledge as low (see Figures 1 and 2respectively).

Figure 1 Level of keyboard skills	Figure 2 Computer literacy skills

56% of the respondents had less than 5 years of teaching experience by face to facemode and 76% of them had less than 5 years teaching experience at the distanceeducation/open learning mode. While 52% of them self nominated for the training, and 48%of them were nominated by their supervisors.

Results and Discussion

The following section presents the results of the quantitative and qualitative analysis.

Quantitative analyses

Table 1: Frequency compilation of the responses to items on Section Bof the ACTPEQ

The results of the frequency analysis of the responses of the workshop trainingparticipants to the items on Section B of the ACTPEQ are as shown in Table 1. A criticalexamination of the results indicated amongst others, that the participants returned a meanscore of 4.0 and above on 17 items (see Table 1, items 10, 11, 14, 15, 16, 17, 29, 30, 34,36, 37, 41, 42, 48, 49, 50, 55). A composite summary of these positive responses from theparticipants of the training workshop was that they are particularly pleased with threeaspects of the training: what they experienced during the training program; theinteractivity characteristic of the audiographic equipment; and the learning environmentrequirements for the use of audiographic conferencing. Although they felt confident to useaudiographics (item 23), they felt they have also acquired the skills to prepareaudiographic lessons (item 21). The positive responses given for all the sub-items on item20, which focused mainly on the training session, indicated in the main that they wouldhave liked the training to be prolonged to enable them to acquire more skills and becomemore familiar with the technology (see Table 1 items 20i to 20vi).

A further extension of the analysis was to examine what differences, if any, hadoccurred in their expectations prior to and immediately after the training. In effect, theitems on two sub-scales (expectations of audiographics conferencing prior to training, andexpectations of audiographics conferencing prior to training immediately after training)were compared. The comparison was undertaken by examining their graphic representationsfollowed by a comparison of the differences in their mean scores using the t-teststatistics. The results are as shown on Table 2 below.

Table 2: Comparison of audiographic conferencing traineesexpectations prior to and immediately after training.

Before The Training	Immediately After The Training
t-value = .51, p = .61
t-value = .97, p = .34
t-value = 3.00, p = .01
t-value = -.24, p = .81

Before The Training	After The Training
t-value = -1.31, p = .20
t-value = 3.08, p = .01
t-value = .62, p = .54

The results of the paired t-test comparisons indicated that significant differences(p< .05) were found in their expectations for only two items. They are the itemsdealing with confidence in the use of audiographic conferencing and the difficulty inmastering audiographic conferencing equipment. While their confidence in the use ofaudiographics dropped after training (t-value = 3.00, P=.006), their expectation that theaudiographics will be difficult to learn, significantly dropped after training (t-value =3.08, p= .005).

As mentioned earlier, a number of independent variables were included in Section Awhich sought for biographical details of the respondents. The independent variables weregender, age, qualification, keyboard skills, computing knowledge/skills, level ofteaching, teaching experience in face-to-face mode, and teaching experience in distanceeducation mode. Each of the variables had several groupings.

An analysis of variance was carried out for each variable versus each factor on theitems contained in Section B of the instrument. The results are as shown in Table 3 below.

Table 3: ANOVA of mean differences of selected independent variables in the responses of the audiographic trainees to the various subscales in the questionnaire.

* significant at P< .05.

The results indicated that only two of the analyses were significant. They were theeffect of the different levels of keyboarding skills on their perception during theworkshop (F=4.55, p<.05), and the effect of their face-to-face teaching experience ontheir perception of the learning environment (F=11.03, p<.05). These results seemed toindicate that the differences in their keyboarding skills affected their assessment of thetraining workshop. The more proficient they were with their keyboarding skills, the morepositively they expressed their perception of the workshop. Similarly, the moreexperienced teachers expressed a higher positive assessment of the audiographic learningenvironment than those less experienced.

(ii) Qualitative analyses

Free responses to Section C (a) and (b)

Free responses (a) and (b) pertaining to 'The difficult features of using thisequipment for education and training purposes', and 'The easy features of using thisequipment for education and training purposes', which were found to be unique are listedbelow. There was some correlation in these responses with the computer literate findingall features relatively easy (to master) and those without prior computer knowledgefinding them much more difficult.

Free responses to Section C (c)

These free responses pertained to “What have you gained from this workshop?”

The comments:

Basic understanding of the system
Basic introduction
A new way of presenting lessons
Better computer literacy
More confidence
Confidence
Competence
Experience in designing and holding audiographics workshops.
At this stage an understanding (not comprehensive) of audiographics.
An excitement and desire to use audiographics.
Skills in using PowerPoint and audiographics.
Ability to assess aspects of the technology for use in learning and teaching
It will broaden my teaching strategies
More strategies
New insight into doing a couple of courses myself in the future

Free responses to Section C (d)

The following free responses were given to “Suggestions for improving futureworkshops”. Responses to this section were received from 81% of the participants.

Suggestions:

Some keyboard/compute skills as pre-requisite
More (practical work) on Smart Board
Extra time for course devoted to conferencing techniques
Can't think of anything - was very well done.
Perhaps more time developing slides
Consider a 2-day workshop with “presentations” in afternoon of second day.
“Models of potential teaching techniques/strategies”
“Learn about audiographics through audiographics (understanding the studentsposition)”
It was a great workshop. Thank you!
Need more modelling
Ensure room booked
Provision of tea/coffee & water at midbreak-4 hours is a long time, especially when itis tacked on to the rest of the day. ie. our workshop was 5-9pm Thursday & Friday.
Need more time to get used to all the features.

Summary and Conclusion

This evaluation was primarily focused on audiographic conference training for those whowould continue to use them in their teaching, and to gather information on participants'perception of the value of the training package, the delivery mechanisms, the audiographicequipment, and other factors which affect the learning of the applications of theequipment within a distance education and open learning environment. 26 out of 30participants at the training workshops, located in different sites of the Queenslandaudiographic conferencing, participated in the evaluation. An instrument, The AudiographicConference Training Program Evaluation Questionnaire (ACTPEQ) was designed forparticipants undertaking the training program to gather data for a comprehensiveevaluation of their training program.

The results of the evaluation indicated the following:

• the majority of the participants found the training workshop very beneficial, easier than they had expected, and that computing skills had no relationship with their mastery of the use of audiographic conferencing equipment.
• the participants were pleased with the interactivity characteristic of the audiographic equipment, and the learning environment requirements for the use of audiographic conferencing.
• while the confidence level of participants in the use of audiographics decreased immediately after the training, their expectation that the audiographics would be difficult to learn also significantly dropped after training.
• only the independent variables of keyboarding skills and teaching experience in the face-to-face mode showed significant differences in the responses of the participants with regard to the factors of the actual workshop training and the audiographic learning environment respectively.
• generally the participants expressed the view that they would have liked the training to have been longer to enable them to acquire more skills and become more familiar with the technology.

Regarding the trainees' expectations prior to and immediately after audiographictraining, the following summaries can be made:

Skills and Confidence

• Participants were more positive towards their acquisition of audiographic conferencing skills before the training. Immediately after training, they were less likely to rate themselves as highly in skill areas. These comments apply to skills in preparing audiographic lessons, delivering audiographic lessons, and confidence in using audiographics.

View of Audiographics Use

• After the training, more participants were inclined to view audiographic conferencing as being able to meet the needs of their students. A marginal increase in the number of participants who believed that audiographic conferencing would improve the effectiveness of their distance education was evident after training.

Learning Audiographic Conferencing

• Fewer participants thought that audiographic conferencing was difficult to learn after the training than before the training.

Based on the results, the following recommendations are made:

1. The findings indicate strong support for the need, value and benefit of audiographics conferencing training. Institutions and organisations as a matter of policy should provide training and professional development opportunities for teachers using audiographics conferencing.
2. Contrary to initial perceptions, users found audiographics conferencing relatively easy to use. Furthermore, low computer literacy skills were not a barrier to users once people had received training. Teachers/lecturers regardless of computer literacy skills should be encouraged to use audiographics conferencing, provided training is available. Furthermore, an awareness campaign should be undertaken to promote that high computer literacy may not be important.
3. Low keyboard skills were considered a barrier to users during training. The training course should include keyboard familiarisation for people with low computer literacy skills.
4. Further longitudinal research work is required and should be conducted in this area to investigate the generalisability of these findings.

References

Anderson, T., & Mason, R. (1993). International computer conferencing forprofessional development: The Bangkok Project. American Journal of Distance Education,7 (2), 5-18.

Blinn, C. R. (1989). Developing telecommunication linkages for microcomputer aidedinstruction. Telecommunication Development Center Research Report No.1. MinnesotaUniversity: Telecommunication Development Center.

Burge, E. J. (1993). The Audioconference: delivering continuing education foraddictions worker in Canada. Journal of Alcohol and Drug Education, 39 (1), 78-91.

Clark, G. (1989). Distance education in United States schools. Computing Teacher,16 (6), 7-11.

de Vries, L., Naidu, S., Jegede, O. J., & Collis, B. (1995). Online professionaldevelopment: An evaluation study. Distance Education, 16 (1), 157-173.

Ehrmann, S. C. (1990). Reaching students, reaching resources: Using technologies toopen the college. Annerberg/Corporation for Public Broadcasting project. (ERICDocument No. ED327171).

Gilcher, K. W., & Johnstone, S M. (1988). A critical review of the use ofaudiographic conferencing systems by selected educational institutions. MarylandUniversity, College Park. (ERIC Document No. ED313003).

Gooley, A., & Towers, S. (1996). Turning ocean liners: Managing interactivetechnology innovation. Proceedings of the 3rd International Interactive MultimediaSymposium, Perth. 149-159.

Gooley, A., & Jegede, O.J. (1995). A longitudinal study of audiographicconferencing system in open and distance learning in Queensland. In F. Nouwens (Ed.). CrossingFrontiers. Proceedings of the 12th Biennial Forum of the Open and Distance LearningAssociation of Australia, September, Vanuatu, 17 - 24.

Hughes, A., & Priestley, J. (1992). The SOLS TICE project: satellite television andaudioconferencing in continuing professional development for LIS staff. InformationServices and Use, 12 (3), 263-274.

Jegede, O. J.& Gooley, A. (1994). A front-end evaluation of audiographicconferencing system in open and distance learning in Queensland. Proceedings of theFirst International Conference on Open Learning, November, Brisbane, 175-180.

Lundin, R. (1994). Flexible learning processes for the delivery of professionaldevelopment activities of principles: A national report to the Australian Principals'Associations Professional Development Council (APAPDC) for Project 1. Brisbane: QueenslandUniversity of Technology.

Martin, D. J. (1986). Audiographic conferencing in small rural schools in New Yorkstate: A mode for developing shared teleconferenced services between rural schooldistricts. (ERIC Document No. ED272336).

Murphy, K. L. (April, 1988). The integration of teleconferencing in distanceteaching. Paper presented at the Annual Conference of the Association of EuropeanCorrespondence Schools. Istanbul, Turkey.

Robson, J. (1991). Remote schooling and information technology (A guide forteachers). Canberra: Australian Catholic University.

Westerman, D. A. (1990). A study of expert and novice teacher decision making: Anintegrated approach. (ERIC Document No. ED322128).

Wilsman, M. J. (1988). Distance education in action: The Wisconsin rural readingimprovement project Report. Madison: Wisconsin Public Radio & Television Networks,Education Services Division.