Jim Woolnough

C. Leah Moore

Andrew Cheetham

John Rayner

University of Canberra

Abstract

Student responses to a tertiary, flexibly delivered physics course are examined. The course was designed to provide secondary science teachers in rural regions of NSW, Australia, with a qualification appropriate to teach senior physics, and is a response to the lack of new physical science teachers currently being trained. Evaluations indicate that it has been highly successful overall, but also reveal something of how participants responded to different modes of delivery. Both students and lecturing staff valued highly the opportunity to experience a more traditional mode of teaching and learning provided through an intensive residential component of the course.

Introduction

The Graduate Certificate of Physics (GCP) described in this paper is a response to two recent educational phenomena. The first is the potential shortage of teachers in the physical sciences and mathematics, and the second is the rapid development of increasingly sophisticated platforms for online delivery. The original image of distance education providing opportunities for isolated students who are unable to access normal educational facilities is giving way to an increasing desire for more convenient educational offerings. Where distance education may once have been seen as a less desirable and more expensive alternative there is some suggestion today that the flexible delivery of courses may, in fact, be cheaper and even preferable to traditional models. Certainly, we have moved from an era when distance learning was viewed as a last resort to one where it is seen as presenting opportunities for new teaching and learning markets. But have we reached the stage where science courses can be delivered through flexible/distance modes as effectively as through traditional, on campus courses?

In a recent review of the effectiveness of online education Jung & Rha (2000) suggest that reduced costs and increased revenue have been major factors in the drive towards online education, but many studies purport to show that online delivery modes produce educational outcomes are generally as effective as more traditional face-to-face modes. Some observations (Inglis, 1999) suggest that online education may produce improved educational outcomes through wider access to a variety of multimedia resources and information combined, surprisingly, with increased opportunities for interaction with other students and instructors.

Russell (2001) has compiled a collection of some 300 research studies claiming to show that there was ‘no significant difference’ between students’ learning through online environments as compared with face-to-face learning environments. This site also links to a selection of reports that do suggest some difference in learning outcomes between these two groups; in most cases with online learning producing slightly better results. In a review of the literature on the ‘no significant difference’ phenomenon, Ramage (2002) questions the validity of these studies, in that there is so much variation in ‘traditional’ courses that it has not actually been possible to exclude a whole host of variables that could be operating in any or all of these studies. He concludes that there is no study, no evidence of any kind that categorically proves that technology does not impact learning in some way, positively or negatively.

It has often been assumed that missing out on traditional experiences such as lectures compromises the quality of the learning by distance experience. Biggs (1999:113) describes how, at the institution where he taught, parity between internal and external students was maintained by denying the internal students access to the external lecture notes, to make up for the advantage the internal students had in being able to attend the on-campus lectures. In fact, numerous studies suggest that distance education often seems to be as effective as and sometimes more effective than traditional modes (Jung & Rha, 2000).

Some recent studies of courses that have been delivered online suggest a high level of acceptance by students (Chang & Fisher, 1999; McConnell & Shoenfeld-Tachner, 2002). The latter study involved a science course (in histology) in the US. Students in this course judged it to be ‘readily accessible and at least equal in academic rigour to comparable on-campus courses.’ One of the main issues in the development of the course described in that paper was the incorporation of appropriate laboratory work. How did the course in histology deal with this? It was left out. Students accessed excellent images of histological sections via the website but did not handle a microscope. In fact one student response stated, “The labs online were extremely helpful and fast. You didn’t have to waste time with a microscope and slides, they [images] were there for you.” The authors suggest that the field of ‘microscopy’ is distinct from ‘histology’ and chose to set up an alternative, optional, microscopy course that was offered, separately on campus. Our belief that a substantial laboratory-based experience was needed was the main reason for incorporating a residential component into the GCP.

This paper explores the tension between flexible/distance teaching modes of delivery and the more traditional teaching and learning environments provided during the residential component of this course.

Course structure

This GCP was designed to provide existing science teachers with an appropriate qualification to teach senior (Year 11 & 12) physics in NSW government schools. In the first year of delivery (2002) there were 27 students, all sponsored by the NSW Department of Education and Training. The second and third cohorts had around 23 students each. They have all been qualified secondary science teachers and most taught full-time during their study. More than half the students are from rural schools. The original development of this course was a specific response to the needs of teachers in rural schools, but a third of the students come from the Sydney metropolitan area and chose the online course in preference to an on campus course that was offered at the same time. This probably relates more to the current drive for convenience over a more traditional educational experience rather than selection for educational reasons. Across the 3 cohorts around 50% of the students have been female. Neither teaching staff nor students had any previous experience of distance education before this course was initiated.

The course provides a Graduate Certificate of Physics, comprised of two physics subjects, comparable to first year university physics, and one subject dealing with pedagogy and assessment in physics.

The course is being delivered over two semesters and combines three delivery modes:

• A website. The University of Canberra has adopted WebCT as the preferred delivery platform for all online delivery as well as for support of on campus courses.

• A physics textbook (Giancoli, 1998). This text provides a broad introduction to physics and is not calculus-based. It was chosen because a substantial WebCT site, which is intermeshed with the University site, supports it. Through this site students can access extensive online tests, animated quizzes, applications, and assignments.

• Two intensive residentials (5 days in each semester). These involve a series of lectures, workshops and laboratory sessions.

In general, the textbook provides what textbooks have always provided in undergraduate science courses but, in this case, is intended to fulfill a central teaching role in the absence of lectures. The website provides directions as to what section of the text book to read from, further readings, links to websites, tests and quizzes, bulletin boards allowing online discussion and access to other students and tutors. WebCT sites also have their own secure internal mail systems, which allow private student/student and staff/student communication.

Challenges for Delivering Physics Flexibly

Concern has been expressed with respect to the capacity of flexible delivery to cater for some types of learning outcomes in higher education settings. Toohey (1999:118-120) suggests that the delivery of technical and conventional knowledge through flexible delivery modes is fairly unproblematic, but that deeper understanding of complex concepts that often run counter to what is learned from common experience provides some challenges. The solutions she suggests tend to emphasise the need for quality interactions, usually between ‘tutor’ and students.

An important development in available platforms for delivering courses online is in their capacity for interactions, both student/student and student/teacher. WebCT provides bulletin boards that allow asynchronous communications throughout the semester. It also allows the lecturer to track and monitor these interactions. It would appear that we are fast reaching the stage where this facility is providing access comparable to that provided through traditional courses. In a review of a wide selection of courses provided flexibly Beattie & James (1997) found that some staff felt that the level of interaction was actually better than usual and that the gap between lecturer and student was actually reduced. Several students reported that they had a more ‘human’ relationship with their lecturers. Questions were still raised, however, about the quality of feedback from students in terms of their learning and understanding.

There is evidence that the types of interactions that occur through online discussion groups and bulletin boards are different to the types of interactions that occur in normal tutorials. Hewson & Hughes (1999) ‘were surprised by the slow motion nature of the classroom dynamics of our online group’. Contributions to the bulletin boards tended to be more formal and less spontaneous than in live groups, and pauses that could normally last a few moments could last for days online. Our observations in other courses delivered through lectures and tutorials but supported by a website with bulletin boards support these observations overall. Interestingly, we have found that the types of issues raised on the bulletin boards can be quite different to those raised in live tutorials, as if the distance and disconnectedness of the online environment allows students to broach issues they would hesitate to raise in class. Perhaps this is because they are not related to the current topic of discussion or because the students would be embarrassed to raise them.

The authors felt that the issue of depth of learning was an important one for this physics course, particularly for teachers, who will themselves need to deal with the same issues with their own students. Toohey (1999:9) and Biggs (1999:16) both describe the usefulness of considering students in terms of whether they have a ‘deep’ or ‘surface’ approach to their learning. This is particularly relevant in the delivery of a course online where the ‘deep learning’ approach may pose particular problems. Do we expect our students to be ‘deep’ or ‘surface learners’? Indeed, two of the factors that Biggs (1999:13) suggests push students towards a surface approach are evident in this cohort, namely, time stress (most students are full time teachers) and lack of choice over subjects (the Graduate Certificate has three prescribed subjects). In most senses, as we would hope of practicing teachers, the students in this course have been extremely serious about their studies, and go to great lengths to ensure that they have a sound understanding of all topics, but their perception of depth has not always exactly matched ours. While the students, as expected, were highly focused on being well-prepared to teach specific topics at senior level, our perception of depth extended to the pursuit of topics central to physics but not necessarily covered at senior secondary level in the current NSW curriculum.

Perhaps the tendency to see our traditional teaching methods through rose-coloured glasses suggests that while we should be alert to challenges and deficiencies in our courses delivered flexibly we should be wary of setting the bar too high. When Toohey (1999) suggests that high quality interactions are needed to ensure deep learning of difficult concepts it is easy to make the assumption that traditional courses actually provide these. An evaluation of students in traditional first year physics courses in universities in the US, Australia, and Thailand, suggested that student understanding of force and motion concepts show very little development throughout the course. Comfortingly, Australian students tended to enter these courses with a better understanding, and consequently performed better on the evaluation test at the end of their tertiary course (Thornton & Sokoloff, 1998; Johnston & Millar, 2000; Emarat et al., 2002).

Results

How well has flexible delivery worked?

Towards the end of the first semester students were provided with the opportunity to evaluate the course through an anonymous online questionnaire. This is a standard feature of the WebCT platform. While it is possible to monitor which students have responded to the questionnaire, it is not possible to match responses to students. As part of this questionnaire students were asked to assess the contribution of each of the components of delivery of the course to their overall learning.

The specific question asked was, “Now that you have experienced the full range of modes involved in this course, could you estimate the relative importance of each mode in your learning of physics and physics teaching?”

Figure 1. Student perceptions of how much each component of the course contributed to their learning. The first and third cohorts of students were evaluated. [For cohort 1; n=21. For cohort 2: n=9. For cohort 3; n=17]. Each cohort had 23-27 students overall. Error bars are shown except for cohort 2, which represented a small sub-sample of the whole cohort.

There was significant variation in how students in the different cohorts responded to this question. The first cohort of students valued the residential significantly above the other two modes (Figure 1). This trend seems to have been evident for the second cohort as well, although only a small proportion of these students were evaluated. However, the third cohort of students responded much more positively to the text book with a reduced response to the residential component. It is very difficult to account for this change. There had been some restructuring of the problem-based tests for the third cohort that may have encouraged students to refer to their text book very early in the term. It is possible that, for a distance education course such as this, small adjustments in how material is presented and handled produce considerable changes in how students relate to the course materials. This would point to a considerable fragility or tension in the relationship of students to the course that conveners should be aware of.

It was also evident that the lecturers valued the residential no less. Classes and workshops were scheduled daily from 9:00 until 5:30 and effectively involved most of the lecturers teaching beyond their prescribed load to achieve this. This suggested some urgency on our part to extract all possible value from this component. It is noteworthy that the students, in the rest of their evaluation, provided very positive feedback on nearly every session despite the excessive workload (Table 1). Over a range of questions about various aspects of the residential the average Lickert scale score was 2.1, where 1 represented ‘excellent’ and 5 represented ‘unsatisfactory’.

Biggs (1999:115) has suggested that although students (and teachers) can overcome their initial fear of electronic technologies “they and their teachers still feel the need for face-to-face contact”. In an address at the Uniserve Science Annual Conference at the University of Sydney (April, 2002) Beryl Hesketh spoke of the danger of “cigarette courses”, where trainers and students happily agree on, and become addicted to, enjoyable but less than exemplary teaching and learning strategies (Hesketh, 2002). In our course it is difficult to resist the conclusion that there was some collusion between teaching staff and students in honouring the face-to-face component of the course.

This is not to say that this decision was not well founded. An original concept of the course was that the delivery of a satisfactory laboratory component would require such a delivery mode. To test this comparison was made between student responses to the theoretical and practical components of the residential for the first cohort of students (Table 1). During the week the teaching staff assigned the same amount of time to theoretical lectures and workshops as to laboratory work. These sessions included lectures on physics topics and feedback sessions on assessment completed earlier in the term. Students responded equally well to both types of sessions. Despite the extensive workload the average response to all sessions was ‘very good’. There was no significant difference between the responses. The high variation in the responses to the lectures and tutorials was mainly due to one outlier, a one-hour session on assessment that scored 3.95 (not so good). This was the least structured session in the whole week and at least demonstrates that the students were prepared to be critical of a session they didn’t like. The results from the third cohort of students were very similar. The overall ranking was 2.1 with no significant difference between lectures or laboratory sessions. Despite ranking the residential lower overall in terms of significance to their learning, they still ranked the individual residential sessions highly.

Impressions of Staff and Students

The students in this course have expressed opinions in a variety of forums. As well as providing comments formally through the online evaluation they have commented through the bulletin boards as well as in person at the residential. Given the recognized importance of effective interactions in quality learning it was interesting to observe the nature of the interactions as they developed during the first semester of the course. Very few of the students had met before the course. At the commencement of the course most of the students visited the campus for one day of orientation. This seems to have been significant in the development of student/student interactions online. Of particular interest was the development of a relationship between the students and the lecturer who managed the website (LM). Despite the fact that they had least interactions with her at the orientation, by the time of the residential, two months into the course, they had developed a particular trust in her. This became evident, as students would approach her rather than the other lecturers about their concerns. In this case a definite relationship had developed through online interactions. This supports the observations of Beattie & James (1997) that some staff have experienced improved contact with students in online courses. At the residential more than one student asked if that particular lecturer was going to be at all the sessions, despite the fact that this was not a part of the course that she was particularly involved in.

There was a definite feeling among staff and students that the residential would represent a significant step forward in the course overall in terms of student understanding of topics and concepts. The lecturers took comfort in having the students in front of them and both staff and students seemed relieved at having the capacity for real-time interactions. Both staff and students were in agreement that as many difficult theoretical topics as possible should be covered while we had the chance. It was in response to the evaluation of the first cohort that we extended the orientation period at the beginning of the course from one to two days, including some lectures on topics that would be introduced early in the course. In one respect we felt that students would get to know each other better in preparation for their future interactions on-line, but the main driving force was the wish of the students to have some topics introduced through lectures as early as possible.

Table 3. Student written comments from the on-line evaluation that relate specifically to face-to-face lecturing.

Tables 2 and 3 show some of the written responses of the students in the evaluation. In light of the generally high approval ratings of nearly every session it would seem that;

• In general students appreciated the opportunity for direct contact.

• Students were keen to cover as much material as possible during the residential.

• Students were seeking even more experiences of practical work.

• Students were prepared to put in long hours to achieve the above.

• Students tended to expect material directly focused on NSW syllabus to be covered.

• Students were concerned with how the physics concepts would be related to the classroom

• Students sought material on contexts and applications, beyond what was available in the text.

• Students showed many signs of being ‘deep learners’, in terms of their diligence, engagement and perseverance, but their perception of depth did not necessarily extend to topics outside their own syllabus or with issues of a general pedagogic nature.

The last point raises another important issue that is not the focus of this paper. That is, the tension between ‘education’ and ‘training’. There has been some discrepancy between the students’ desire to have this course recognized as a qualification in physics, in a general sense, and their frequent requests for ‘only topics relevant to the syllabus’ being covered. This is an issue that should be investigated further.

We would conclude that the residential, which provided a close approximation to a traditional learning environment, was accepted by both teaching staff and students as providing essential components of the course beyond the laboratory and practical work. These included a range of material relating to both theoretical knowledge in physics and to applications of physics. Smith (2001), in a study of student learning preferences at an Institute of Technical and Further Education (TAFE), has reported that “[older] students may feel more comfortable in a traditional instructor-led learning environment” than in a flexible delivery mode. Comfort in and enthusiasm for the traditional mode seems to have been a feature of our cohort.

Conclusions

One of the major issues raised in this paper is the tension between flexible delivery and traditional models of tertiary education. Both students and lecturing staff in this course have been drawn to some aspects of face-to-face delivery. This has occurred in the area of laboratory work, which is not unexpected, but also in dealing with a range of more complex concepts and applications. Other studies have suggested that these are the areas where traditional, on campus teaching may be preferable, either because of the need for specialized equipment or because of the need for effective interactions.

There are a number of possible motivations for instigating a flexible delivery approach.

1. Students are isolated. This was the case for more than half of the students in this course.

2. Students are busy and prefer a more convenient approach. A third of our students came from the Sydney metropolitan area yet chose a distance course based in Canberra rather than a local alternative.

3. The flexible delivery modes may be perceived to provide better learning experiences than traditional modes. This was not the motivation for development of this course.

4. Flexible delivery is perceived to be easier or cheaper to deliver. In its first year this course has been staffed at a rate 30-40% above normal. In later years it has continued to be staffed at 10-15% above normal. It is, therefore, considerably more expensive than traditional courses.

A number of factors may influence the choice of traditional delivery modes.

1. ‘Face-to-face’ interactions are desired.

2. The flexible traditional modes may be perceived to provide better learning experiences than flexible/distance modes.

3. There may be a need for specific equipment and facilities.

4. Staff and students may resist flexible modes of delivery because of their own familiarity with traditional modes.

It appears that the main reasons for this course being delivered flexibly relate to the first two points. The isolation of most of our students makes traditional delivery impossible. The convenience factor is also important. But this raises another issue. Tertiary educators are becoming increasingly aware that their students have little time to devote to their studies. Even in undergraduate courses students are likely to work long hours in addition to their studies. Flexibly delivered courses tend to cater for students who are working full time, in this case as secondary teachers. This is likely to have a far greater effect on the success of their studies than the mode of delivery. It also means that the deliverers of such courses tend to make allowances for the limited time students have available for study.

When judging which delivery mode is most appropriate it is important to identify what, specifically, is going to be offered. For example, in a science course the traditional mode would generally involve lectures, laboratory sessions, tutorials, and a textbook. When opting for a traditional approach it is important to identify the value of each component. Do the lectures provide something that is not possible through other means, or can they be easily and effectively replaced by an online alternative? The responses of some of our students suggest that being in the room with an enthusiastic and inspiring teacher is valuable, but is this the normal situation in lectures? Can an online environment provide interactions commensurate with normal tutorial sessions?

We would make two observations based on our experience in this course. Firstly, face-to-face delivery modes have a number of qualities that cannot be replicated in any distance mode, even with the power of modern web-based platforms. But second, we suggest that online delivery can add dimensions that complement traditional delivery modes. In the future many more courses will combine modes in this way, utilizing the best elements of each. It must be recognized, however, that these complementary modes come at a cost, in terms of infrastructure and staff time and education providers must be prepared to bear these costs.

Beattie, K. & James, R. (1997). Flexible coursework delivery to Australian postgraduates: How effective is the teaching and learning? Higher Education, 33, 177-194.

Biggs, J. (1999). Teaching for Quality Learning at University. What the Student Does. Buckingham, UK: Society for Research into Higher Education & Open University Press.

Chang, V. & Fisher, D. (1999) Students’ perceptions of the efficacy of web-based learning environment: the emergence of a new learning instrument. In: Cornerstones: what do we value in higher education? Proceedings, July 12-15, Melbourne, Australia. Canberra: Higher Education Research and Development Society of Australasia.

Emarat, N., Arayathanitkul, K., Soankwan, C., Chitaree, R. & Johnston, I. (2002). The effectiveness of the Thai traditional teaching in the introductory physics course: A comparison with the US and Australian approaches. Proceedings of Uniserve Science Scholarly Inquiry Symposium, April, 2002, 31-37. University of Sydney, Australia. Retrieved July 3, 2004 from http://science.uniserve.edu.au/pubs/procs/wshop7.

Giancoli, D. C. (1998). Physics. Principles with Applications, Prentice Hall.

Hesketh, B. (2002) The science of science teaching and learning. Proceedings of Scholarly Inquiry in Flexible Science Teaching and Learning, Uniserve Science, The University of Sydney, Australia, 2002, 3-6.

Hewson, L & Hughes, C. (1999). An online postgraduate subject in information technology for university teachers, Innovations in Education and Training International, 36(2), 106-123.

Inglis, A (1999). Is online delivery less costly than print and is it meaningful to ask? Distance Education, 20(2), 220-239.

Johnston, I. & Millar, R. (2000). Is There a Right Way To Teach Physics? CAL-laborate, 5, 10-14, Retrieved July 3rd, 2004,from http://science.uniserve.edu.au/pubs/callab/vol5/johnston.html.

Jung, I. & Rha, I. (2000). Effectiveness and cost-effectiveness of online education: a review of the literature. Educational Technology, 40(4), 57-60.

McConnell, S. & Schoenfeld-Tachner, R. (2001). Transferring your passion for teaching to the online environment: a five step instructional development model, e-Journal of Instructional Science and Technology 4,(1). Retrieved July 3rd, 2004, from http://www.usq.edu.au/electpub/e-jist/.

Ramage, T. R. (2002). The "No Significant Difference" Phenomenon: A Literature Review, e-Journal of Instructional Science and Technology, 5(1). Retrieved July 3rd, 2004 from http://www.usq.edu.au/electpub/e-jist/.

Russell, T. (2001). The "No Significant Difference Phenomenon". Retrieved July 10th, 2002 from http://teleeducation.nb.ca/nosignificantdifference/; November, 2004 site moved to http://nt.media.hku.hk/no_sig_diff/phenom1.html.

Smith, P. J. (2001). Technology student learning preferences and the design of flexible learning programs. Instructional Science, 29, 237-254.

Thornton, R. K. & Sokoloff, D. R. (1998). Assessing student learning of Newton’s laws: the force and motion conceptual evaluation and the evaluation of active learning laboratory and lecture curricula, American Journal of Physics, 66(4), 338-352.

Toohey, S. (1999). Designing Courses for Higher Education. Buckingham, UK: Society for Research into Higher Education & Open University Press.