In most Australian universities, student-perceptions-of-teaching surveys are seen to play an important role in assisting staff to improve their teaching. It is often assumed that on receipt of their student survey report, diligent and committed academics will review and interpret their report, identify areas of weakness in their teaching and seek to address them though various means, including identifying suitable staff development activities and accessing print and non-print resources. While some members of teaching staff do manage to find the time to identify staff development workshops on teaching or access relevant books and other resources, many do not because of difficulties in identifying their specific needs as well as their perception of the effort and time involved in locating and accessing resources.
The project 'Linking Feedback from Student Perceptions of Teaching to Active Strategies for Teaching Improvement', funded by a two-year grant from the Committee for University Teaching and Staff Development, is a collaborative project between the University of Adelaide and the University of Western Australia. The project aims to provide an effective way of linking feedback on various aspects of teaching to existing resources that might assist in the development of strategies to enhance student learning through changed teaching practices. Poorly rated items in student survey reports will be linked with relevant strategies and resources through an automated system. Academics receiving feedback reports will thus be provided not only with student ratings, but also areas for development and change would be identified, and available strategies such as workshops, books, videos, CD-ROMs and Web-based resources would be listed. For the project team in each university, this involves auditing and classifying existing resources, establishing databases and linking selected student feedback items with resource databases.
This paper provides details of the project and a progress report on what has been achieved and learned in the first six months of the project.
Current student survey reports are intended to enable teachers to identify areas of strength and weakness in their teaching and learning strategies. In theory they will take note of the items with low ratings in their report and then set out to improve their teaching through a variety of means, including accessing staff development opportunities and materials provided by their institutions. However, for many academics it is difficult to find the time to locate and access relevant resources or attend teaching workshops, or even to seek advice on what would best suit their development needs.
In order to address the gap between feedback on teaching and access to relevant staff development, ACUE and OSDS are carrying out a project which aims to provide teachers with timely and relevant strategies for improving their teaching. The project 'Linking Feedback from Student Perceptions of Teaching to Active Strategies for Teaching Improvement' has been funded by the Committee for University Teaching and Staff Development for 2000 and 2001. This paper describes the project and provides a work-in-progress report of the first six months of the project.
The project has four major components:
As each institution has a large collection of survey items (thousands), the project is limited to only items that appear frequently in student questionnaires. Only the three lowest rated items will be linked to teaching improvement strategies in order to assist the teacher to focus on areas most needing improvement. In addition, focusing only on three areas is likely to contain the size of the task so that it seems manageable and achievable in the context of full academic workload.
Each institution has a project team comprising staff with a range of skills and knowledge, including project management, student evaluation of teaching, teaching and learning, academic staff development and programming.
|ACUE Project Team:||Robert Cannon, Director|
Helen Limburger, SET Manager
Gerry Mullins, Senior Lecturer
|OSDS Project Team:||Owen Hicks, Director / Barbara Black, Acting Director|
Elisabeth Santhanam, Senior Research Fellow, Evaluation of Teaching Unit
Kenn Martin, Project Officer
It is intended that the detailed classification structure for resources will be developed on an ongoing basis by identifying keywords for each resource that is linked with an item. For example, a general text on teaching and learning in higher education may contain an excellent chapter on student participation in tutorials which could be linked with the survey item 'I have been encouraged to participate in class'. The resource collection database would have the key words 'tutorial participation' added for that particular text, and the list of keywords would grow as additional chapters or pages of the text were identified as resources for other items. In this way a detailed classification system will be built up over time as resources are identified for more and more survey items.
One of the most difficult tasks has been determining which three of the many resources and activities available would be recommended for a particular item. While strategies are intended to be relevant for all teaching staff, it was recognised that they should not be too generic. For example, encouraging students to take an active part in class may require different strategies in lectures, tutorials, clinical sessions and practical classes. It was also agreed by the project teams that the strategies should not be merely a list of teaching techniques or tips. The following has been developed as a working guide for choosing a breadth of resources and activities for each item:
The project teams are also concerned with the quality of the strategies being provided, i.e. whether they are the 'best' of the resources and activities available. At a recent meeting of the OSDS project team, it was agreed that each item should be researched by two team members separately, who would then exchange the strategies identified and rank the top three. These would then be subject to a critical review by the ACUE team to further ensure quality. Items researched by ACUE would be reviewed by OSDS in a similar way.
Careful consideration is being given to the presentation of the section of the report containing teaching and learning strategies. It is important that those receiving the revised feedback reports for the first time are not made to feel deficient in their teaching or that they scored poorly in the student surveys. The introduction to this section of the report will need to make clear to recipients that suggestions for strategies to improve their teaching are based on their three lowest scores, regardless of how high or low their mean scores are. A teacher receiving high scores for all items would still receive suggestions for improving their teaching. This approach is a reflection of the academic development focus of the project and the need for ongoing improvements to teaching.
We are also reviewing whether the provision of feedback advice should be an option that is selected by the staff member when they order an evaluation to avoid the possibility of antagonising staff with gratuitous advice.
In the next couple of months, reprogramming of report-generating software will commence with a view to an initial testing of the automated linking of selected survey items with the database. It is intended that enhanced reports with strategies for selected items will be available to a limited number of teaching staff by early next year. Feedback will be sought from SPOT and SET users in order to identify any changes required before proceeding with the provision of enhanced reports for all commonly used items later in 2001.
In 2001 profiles will be developed of the frequency of occurrence of recommended strategies and the frequency of accessing of academic development activities and resources at each university. A dissemination package will be developed towards the end of 2001.
A further work-in-progress paper will be presented at the 2001 HERDSA conference.
Brinko, K.T. and Menges, R.J. (Eds) (1997). Practically Speaking - A sourcebook for instructional consultants in Higher Education. Stillwater, Oklahoma: New Forums Press.
Marsh, H.W. and Roche, L. (1993). The use of students' evaluations and an individually structured intervention to enhance university teaching effectiveness. American Educational Research Journal, 30(1), 217-251.
Marsh, H.W. and Roche, L. (1997). Making students' evaluations of teaching effectiveness effective. American Psychologist, 52(11), 1187-1197.
McKeachie, W.J. (1979). Student ratings of faculty: A reprise. Academe, 65(6), 384-397.
McKeachie, W.J. and Lin, Y.G. (1975). Use of standard ratings in evaluation of college teaching. Final Report to National Institute of Education, Ann Arbor: Department of Psychology, University of Michigan.
Moses, I. (1988). Academic staff evaluation and development. St Lucia: University of Queensland Press.
Murray, H.G. (1997). Does evaluation of teaching lead to improvement of teaching? International Journal for Academic Development, 2(1), 8-23.
Trigwell, K. and McKenzie, J. (1995). How can highly rated teachers assist with teaching development? in Brew, A. et. al. (Eds), Judging the quality of teaching: Approaches to university-wide student feedback. Sydney: Centre for Teaching and Learning Monograph Number 1, University of Sydney.
Wilson, R.C. (1986). Improving Faculty Teaching - Effective use of student evaluations and consultants. Journal of Higher Education, 57(2), 196-211.
|Contact details: Barbara Black, The University of Western Australia|
Phone (08) 9380-3845 Fax (08) 9380-1156 Email email@example.com
Please cite as: Black, B., Cannon, R. and Hicks, O. (2001). The missing link: Developing the nexus between student feedback surveys and development for teachers. In L. Richardson and J. Lidstone (Eds), Flexible Learning for a Flexible Society, 69-74. Proceedings of ASET-HERDSA 2000 Conference, Toowoomba, Qld, 2-5 July 2000. ASET and HERDSA. http://www.aset.org.au/confs/aset-herdsa2000/procs/black.html