ASET 2002: Tuovinen - the National ICT Education Enquiry 2001

The National ICT Education Enquiry 2001

Juhani E. Tuovinen
Charles Sturt University

The Australian federal government conducted a national study of the ICT education in Australian universities in 2001. The study investigated how ICT educators work in universities, especially their educational innovations and the dissemination of the innovations, how the graduates of the tertiary ICT education programs viewed their preparation for the workplace, and how the employers of the graduates viewed the existing ICT education.
The study group surveyed existing literature in these areas, surveyed both the graduates and employers and sought information from the practising ICT educators in all Australian tertiary institutions teaching ICT subjects. The ICT educators' views were obtained by inviting them to local working conferences where the research team worked with them to elicit their opinions and the develop ideas in depth.
The results of the study provided information from the perspectives of the ICT educators, the recent graduates, and the ICT employers. Based on these findings recommendations for the government, the universities teaching ICT courses and the employers were formulated to improve the various facets of the Australian ICT educational program.
The recommendations fall into the following categories: employers' views of ICT graduates, graduates' views of their ICT education, ICT educators' views on teaching and learning innovation in ICT education, teaching and learning innovations in ICT education, the dissemination of ICT educational innovations and the educational evaluation of ICT teaching programs.

Introduction

Higher education providers are currently under pressure to review traditional modes of teaching and learning and to develop new educational products. This pressure comes from a number of sources, which together form the broad context of educational innovations. The literature suggests that these sources include

The increasing role of new technologies and globalisation
The changing profile of the student body and student needs
Variations in graduate employment destinations
The changing needs of industry
Increasing commercialisation of higher education and government directives for more efficient delivery
Calls for more effective teaching and improved learning outcomes

In January 2000, the Minister for Education, Training and Youth Affairs established the Australian Universities Teaching Committee (AUTC), a national body aimed at improving Australian university teaching and learning. The Committee then commissioned a small number of projects to investigate the ways in which issues of teaching and learning have been approached in particular discipline areas. The ICT-Ed Project was set up to investigate ICT education.

The ICT-Ed Project included three main data collection phases, focusing on the perceptions of educators, employers and students. The first of these started in late February 2001 and focused on educators' perceptions. The main aim of this phase was to determine how university teaching staff perceive the pressures listed above and what role these pressures play in the innovation process. In this phase, preliminary data on current teaching and learning initiatives was also collected. More innovation related data was then gathered throughout the year.

National mini-conference program

To facilitate the first phase of data collection a mini-conference format was devised (Hurst, et al., 2001, Lynch, Collins & Markham, 2001). This format is described below. Mini-conferences were run in each capital city and via a video link with Darwin. Mini-conferences were designed to maximise the cooperative input from the ICT departments and faculties across Australia so that the final report will be seen as belonging to the national academic area.

Aims of the mini-conferences

The mini-conferences were designed to meet the following aims:

To identify ICT educators' perceptions of the main factors driving educational innovations and best practice in ICT education
To identify current teaching and learning initiatives aimed at improving ICT education
To identify ICT educators' perceptions of the main issues currently facing ICT education
To identify issues involved in the dissemination of information on educational innovations and best practice in the discipline

Target group

The target group for participation in the mini-conferences was tertiary staff teaching in Information and Communication Technology-related areas, especially if they were involved in teaching and learning initiatives. Subject areas included those involved in information systems, computer science and software and computer engineering. Participation was open to all teaching levels and any auxiliary staff involved in teaching and learning initiatives. Each mini-conference could accommodate up to 14 participants.

Format of mini-conferences

The mini-conferences ran from 10am to 4pm on designated days in each capital city. Workbooks provided by the researchers guided the proceedings of the mini-conferences. ICT educators participated in discussions on issues relating to ICT education and presented information on their own teaching and learning initiatives. They recorded their ideas in the workbooks. The proceedings were also audio- and video-recorded, and the audiotapes were transcribed producing a set of text files which could be examined at length.

Industry survey

The second data collection phase was focused on the perceptions of employers of ICT graduates. 3673 surveys were distributed nationally, and 523 responses were received, i.e. a response rate of 14%.

This phase used a questionnaire to ascertain the following:

The skill/knowledge areas that are important to employers of Australian ICT graduates
The degree to which employers are satisfied with these skill/knowledge areas in Australian ICT graduates
The degree of ease/difficulty with which employers hire graduates with adequate skills/knowledge
The career prospects of Australian ICT graduates once hired
The extent of university/industry collaborations in curriculum development and review

Graduate interviews

There is relatively little data on graduate satisfaction with their courses and the effect of this on their job performance. To supplement this meagre information a small number of graduates (N=13) was interviewed to collect in-depth information on these issues.

Common themes were identified in the graduates' stories, such as early exposure to computers, pursuit of intrinsic rewards, self-teaching, entry to industry prior to the completion of a degree, finding employment and life-style preferences. In their reflections on the value of their university degrees the graduates were asked to reflect on the value of the non-programming subject(s) and industry experience, what they gained from universities and suggestions and advice for computer education departments.

Results and discussion

Employers' views

Employers were found to be generally satisfied with their ICT graduates. Industry sees graduates as having a number of obvious deficiencies but these are irritants, or squeaky wheels, rather than being significant decision-making factors. These irritants have been reported in different papers over a 10-year period and are not isolated to graduates from ICT disciplines.

A review of the literature and data collected in this project suggest that the existence of an ICT skills shortage is debatable. More labour market analysis is needed if ICT departments are to have a clear idea of the career environment of their graduates. There is a need for further research to develop a clearer picture of the structure of the ICT industry, of what is critical to the graduate needs of employers, and of employers' commitment to career structures for ICT professionals.

The results from the analysis of the employer questionnaire, combined with suggested shortcomings of previous satisfaction studies, point to the importance of building research models that are sufficiently sophisticated to integrate employer behaviour and graduate recruitment and satisfaction, and provide suitable guidance for planning in the employer and ICT education sectors.

In terms of curriculum development, the ICT sector is subject to the vagaries of changing technologies, ideas and, to some extent, fashions. ICT departments should have depth in their degree structures, which will allow them to readily shift direction as new waves pass through. Focusing on skills training would lead to medium term staffing problems. It would also lead to broad human resource problems as changing technology makes redundant those who have not been educated for adaptation and change.

Generally, employers of ICT graduates have very little contact with university ICT departments. Research needs to be undertaken to ascertain the willingness of industry to support post-graduate specialist training that reflects the changing nature of the industry and its technology.

Graduates' views

Little research has been published on ICT graduates' experiences and attitudes after they have entered the workforce. What does exist is not very useful in terms of curriculum development and educational innovation.

The graduates in our survey indicated an early exposure to computers mainly through their fathers in their homes. They reported the importance of the fun and sense of achievement with computers. In fact, most had taught themselves to program or configure computers, and they valued self-teaching over university degrees, due to expected obsolescence of the university course material.

Many had taken up employment prior to finishing their degrees and this often reduced their motivation to finish their courses. These positions were mostly obtained through family or social contacts, or on recommendations. Most of the people were interested in a particular lifestyle rather than power and money.

The views on the value of the non-programming subjects was mixed. Most did not value such subjects during their courses, but some had come to realise such subjects' value later, e.g. project management, after getting into work that required such skills. The views about the value of project group work were not uniform. Some students had not enjoyed their university group work, or felt that it was poorly organised, but others saw the importance and value of such learning tasks once they had to work in the industry.

The main three benefits the students' gained from their university degrees were an overview of the ICT disciplines and the ICT industry, learning skills and networking opportunities. Their advice to the University ICT departments was that some content included in the courses was out of date or irrelevant to industry, the degrees often lacked integration across subjects and the ICT departments did not provide sufficient career advice.

ICT Educators' Views

The analysis of the mini-conference data identified many factors seen by educators as driving or inhibiting educational innovation. Most of these factors are relevant across the sector regardless of state or territory. However, some variation is suggested between universities in the relative importance of particular factors.

Driving factors

Factors seen to drive educational innovation in ICT education include the personal motivation of teaching staff, changes in the student population, changes in the scale of teaching, the push from administrators for flexible delivery and the development of new content areas and the availability of new tools.

Personal motivations discussed have included individual educators' desire to be better teachers and to be seen to be at the forefront in their fields. The desire to be a better teacher has been described as driving responses to observed problems, such as first year transition problems, student (dis)satisfaction, plagiarism, equity and access issues, and unsatisfactory learning outcomes. Unsatisfactory learning outcomes included the failure of students' skills to meet the needs of industry. Both student (dis)satisfaction and first year transition problems were linked to the larger factors of the changing student population and the changing scale of teaching.
Changes noted in the student population have included more diverse language/ educational background and more diverse experience and skills. The increase in diversity of language/educational background of students was seen by participants to be linked to the growing number of International Students and Government/university policies on enterprise funding and fee paying. The increase in the experience and skills of students appeared to be linked to an increase in the number of enrolments. This, in turn, can be linked to Government/university policy.
Changes noted in the scale of teaching included larger class sizes and fewer contact hours. These changes have also been linked to Government/university policy and push to save money.
Participants have reported being under pressure to move towards more flexible delivery of courses. This push has been seen as complex and has been linked variously to money saving, the availability of new tools, "Internationalisation", growing competition between universities, student needs/demands and fashion. Both "Internationalisation" and competition between universities have been linked to Government/university policy on enterprise funding.
The development of new content has been linked both to the rapid development of technology and to students' demands to learn about the latest technologies.
As with the development of new content, the availability of new tools has been linked to the rapid development of new technologies, particularly the Internet and related tools. Educational fashions and commercial suppliers of tools have also been discussed in relation to this factor.

Inhibiting factors

Factors seen to inhibit educational innovation include professional and personal risks involved in innovation, educators' lack of skills and knowledge, lack of support from management, lack of academic freedom, rapid technological development, student resistance, the increasing scale of teaching and the diversity of the student population.

Participants have reported both professional and personal risks involved in trying something new. Linked to this were issues of intellectual property, job security and colleagues' perceptions.
Both lack of skills with technology and lack of pedagogical knowledge among teaching staff have been reported as inhibiting innovation. This has been linked with lack of time to learn new skills and to review educational research.
Lack of support from management has been noted in terms of lack of resources (funding, workload, human resources), little reward for efforts and lack of senior champions.
Limited academic freedom has been linked to controls/guidelines for production of materials and to intellectual property. These things have been discussed in terms of over-regulation, bureaucracy and the need for accreditation.
Discussions of rapid technological development have included issues of student access, changing fashions/fads and unreliable/inadequate technology, including inadequate bandwidth.
Student resistance has been expressed in terms of students' expectations and resistance to new ways of teaching/learning. Students' expectations have been described as misinformed, immature and conservative, and the feedback received via quality assurance questionnaires has been described in negative terms as inhibiting innovation and good teaching practice.
The scale of teaching (moves towards larger classes and fewer contact hours) has been noted as an issue that both drives and inhibits innovation. Larger scale teaching has been noted as causing particular problems, thus driving particular innovations, while the teaching/learning environment of larger scale teaching has been noted as inhibiting some types of teaching/learning activities.
As with the scale of teaching, the growing diversity of the student population has been noted both to inhibit and to drive innovation, particularly diversity in ability and skills at first year university level.

Teaching and learning initiatives

The impetus for, and aims of, the majority of the teaching and learning initiatives reported here are student-focused, with many responding to needs or problems encountered by students and a desire to improve the teaching of ICT. Other initiatives were motivated from the changing context of university teaching that reduced resources available, increased student numbers and required support of a more diverse student population, or changing needs of employers.

Most initiatives were formally evaluated, however, for a large proportion of these initiatives, standard student feedback questionnaires were the only source of evaluation. Participants recognised the limitations of relying on student feedback questionnaires alone for the evaluation of teaching and learning initiatives. However, given the lack of reward and the lack of skills, many ICT educators chose not to pursue formal evaluations further than the standard student evaluation of teaching questionnaires required by their departments.

It was also observed that many ICT educators underestimate the significance of their teaching practice as innovation and they use informal means to evaluate and improve their teaching, rather than pursuing evaluation methods that better qualify their work for publication as educational research and better demonstrate the scholarship in their teaching.

Of the seventy-nine initiatives presented to the project team, six were specifically highlighted by peers as being innovations, where innovation is taken to involve something that is new or not practised by the majority, that adds value to learner-based outcomes, that has been developed to a point where it is not simply an idea, that is practicable in educational and infrastructure terms and that has been evaluated. The initiatives identified as innovations included:

An investigation that examines the relationship between learning styles and students' methods of usage of a software program, with the aim of developing learning methodology that raise both students' and lecturers' awareness of the impact of learning styles on the learning process.
A network protocol simulator that has been developed for use in undergraduate computer networking courses to allow students to experiment with various aspects of networking within a laboratory environment.
The development of an 'early warning system' to identify those students at risk of failing software engineering subjects. The system is based on research findings suggesting the time it takes a first-year student to complete a lab-based software engineering task is related to their likelihood of successfully completing the whole subject.
The integration of a range of interactive techniques into the teaching of a networking subject.
A first-year core subject on professional computing that aims to develop students' professional and communication skills. Students examine social, ethical and legal issues as they relate to the computing profession and develop communication and teamwork skills via presentations and debates.
The development of an embedded system learning environment that can be used effectively across an entire software engineering program for both on- and off-shore students.

Eleven initiatives were specifically highlighted by peers as being good practice, where good practice is taken to involve something that has been formally evaluated in terms of educational outcomes or teaching outcomes, that is relevant to the student population and that is integrated into a wider educational program. The initiatives identified as good practice that are not already included in the above list of innovations included:

The development of a capstone subject to prepare final-year students for the reality of the IT workplace. This subject involves students working in groups of 10 or more on a software development project. Skills are developed through weekly formal project meetings and end of project presentations and product displays.
The incorporation of the teaching of writing skills into a second-year subject, focusing skills in thinking, research and report and essay writing.
The use of spreadsheet software to demonstrate concepts in topics such as calculus, combinatorics, induction and recurrences. The use of tables, graphs, patterns and colours allows these topics to be taught with minimal algebra.
A third-year software engineering course where students are required, individually and in groups, to analyse, design and implement a software system. Fourth-year students then inspect the third year students' work. Students gain experience in both the development of project specifications and the quality assurance issues.
The monitoring of students' learning during a semester length industry experience program in regular meetings between each student, his/her mentor (teacher) and a member of the industry organisation, so that each student's experience can be tailored according to the possible opportunities in the organisation.
The redesign of a first-year programming subject to promote greater interest in conceptual issues rather than procedural problems. This has involved changing the programming language from C to VB, implementing more practical teaching techniques and encouraging collaborative learning through the use of online forums.

Student satisfaction and educational evaluation

Methods currently used in university departments to assess students' satisfaction with their courses appear to be inadequate for the purpose of continuous improvement.

On the whole, ICT educators are unfamiliar with the principles and techniques of educational research and evaluation methods. This impairs their ability to conduct formative evaluations that provide useful information for the improvement of their teaching. It also inhibits their ability to conduct the summative evaluations that would support the dissemination of educational innovations. A related problem is their lack of skills to assess the innovations of others as reported in the wider educational literature.

A lack of resources and a lack of reward for the pursuit of educational evaluations further compound these problems. Evaluation and dissemination activities are seen to be a very low priority in ICT departments. Consequently, ICT educators, and their departments have difficulty demonstrating that their teaching activities are innovative and their educational benefits.

Recommendations

Improving interactions with the outside world
Recommendation 1	That further research be initiated, within the context of labour market models, to determine the structural differences between those who are likely to employ ICT graduates and those who are unlikely to employ them. In conjunction with this, research is needed into the issue of company size and the consequences of this for the size of the market and the skills needed.
Recommendation 2	That, to explore real-world options on how the university-industry interface can be made permeable enough to help meet industry needs without compromising the broader educational needs of graduates, universities ICT departments be given support in establishing links with industry. In conjunction with this, that exploration of the development of post-graduate, skills-based courses that reflect changes in technology and the industry be undertaken.
Recommendation 3	That, further to the generic approach taken to the current national data collection, universities be given support in implementing graduate follow-up and employer satisfaction studies which focus on local issues and local needs, and which use diagnostic approaches.
Recommendation 4	That the work currently being undertaken by CERG members on student satisfaction form the basis of further work on the development of user-oriented tools and methods for interpreting and diagnosing graduate and employer satisfaction.
Recommendation 5	That research funds be allocated to longitudinal and/or retrospective studies of graduates to obtain an understanding of the career paths through the ICT industries.
Recommendation 6	That university ICT departments, in collaboration with industry, devise programs for the better management of students' expectations of professional life.
Educational innovation
Recommendation 7	That the management of university ICT departments (from now on management) more actively and more visibly encourage and support innovation in teaching through the development of acceptable and practicable systems for the assessment and reward of innovative teaching, and through the allocation of resources.
Recommendation 8	That management support and actively promote the educationally focussed professional development of their teaching staff.
Recommendation 9	That universities continue to give ICT educators the freedom to choose the particular teaching tools that they use, including the development of their own tools.
Recommendation 10	That university ICT departments be encouraged to discuss, and to develop innovative solutions to, the challenges posed by very large class sizes and increasingly diverse student populations.
Educational dissemination
Recommendation 11	That local strategies be developed to enhance the status of teaching in university ICT departments. Such strategies should aim to make more effective links between scholarship in teaching and staff promotion by encouraging and supporting staff in the demonstration of scholarship in teaching establishing systems for management to assess such scholarship communicating the value of teaching through all existing channels
Recommendation 12	That policy on educational innovation and research, as a valid form of scholarship within non-education disciplines, be clarified in order to reduce the perception of educational research as a career inhibitor.
Recommendation 13	That interactive forums for sharing work-in-progress and for receiving peer feedback be encouraged both within university ICT departments and across the relevant disciplines, and that due recognition and support be given to those who organise such forums. In doing this, advantage should be taken of existing relationships between individuals and between institutions.
Recommendation 14	That the teaching and learning initiatives identified in the report as innovative or as good practice be promoted to the wider ICT education community.
Recommendation 15	That efforts to diffuse educational innovations acknowledge and accommodate ICT educators' need to 'make innovations their own'.
Recommendation 16	That efforts to diffuse educational innovations involve two-way interaction between those who generated the innovation and those who plan to adopt it.
Educational evaluation
Recommendation 17	That ICT educators be encouraged, and that support be provided, to evaluate their teaching and learning initiatives through the provision of dedicated services. Emphasis should be given to the promotion of tailored evaluations that focus on investigating links between particular initiatives and educational outcomes.
Recommendation 18	That professional development programs be developed for ICT educators that focus not only on teaching tools and methodologies but also educational evaluation techniques, both those based on student satisfaction and those based on other educational outcomes. Besides training ICT educators in evaluation principles and techniques, these programs should also provide them with frameworks for critically appraising the innovations of others.
Recommendation 19	That a continuous improvement approach be adopted towards quality assurance in ICT schools and faculties so that staff have an effective basis from which to respond to change and input from professional, organisational and industry sources.
Recommendation 20	That university Education departments, in collaboration with academic development units, be encouraged to establish communication links with ICT educators. ICT educators should not have to become inculcated into educational jargon in order to understand functional educational issues - such as basic research design.
Recommendation 21	That university Education departments be supported in the development of materials designed to facilitate the learning of non-education lecturers who are interested in educational theory and practice. There are models for this from other subject specialist areas such as university mathematics teaching.
Recommendation 22	That user-driven evaluation of any professional development materials be carried out. That is, the materials produced for these activities must focus on meeting the needs of ICT educators.

Conclusion

The 2001 investigative phase of the ICT-Ed Project is over. Important findings about the gradates', employers' and ICT educators' perspectives of the national ICT educational program were uncovered. In 2002 the dissemination phase of the project is taking place. In this phase the project in endeavouring to:

Spread the news of the worthwhile educational innovations in ICT education, and
Develop better capabilities in educational evaluation among ICT Educators (this was a major identified deficiency among the innovations and the innovators in ICT education project's first phase).

The group is aiming to achieve these aims by:

Developing a Website which will help to manage the sharing and reviewing of ICT educational innovations
Hosting seminars on ICT Innovations sharing during the second half of the year.

It is anticipated and desired that ICT educators from around the country would participate in these activities in a number of ways:

Post information about their educational innovations to the ICT-Ed Project Website
Review the information about educational innovations on the Website
Become actively involved in the educational evaluation skills development programs
Utilise the content of the ICT-Ed Project Website and other related materials as data in research programs
Collaborate with other interested parties in trying out innovations suitable for their situations.

References

Hurst, J., Carbone, A., Eley, M., Ellis, A., Hagan, D., Markham, S., Sheard, J., Tuovinen, J., Lynch., J. & Collins, F. (2001). Teaching ICT. ICT-Ed Project. The report on learning outcomes and curriculum development in major university disciplines in Information and Communication Technology (draft). http://cerg.csse.monash.edu.au/icted/ictedstage1draft.pdf [viewed 19 May 2002, verified 2 Sep 2002]

Collins, F., Lynch, J. & Markham, S. (2001). The mini-conference as a research tool: Encouraging collegiality among ict educators. Proceedings ASCILITE 2001 Conference, Melbourne, 133-139. http://www.ascilite.org.au/conferences/melbourne01/pdf/papers/collinsf.pdf [viewed 19 May 2002, verified 2 Sep 2002]

Lynch, J. & Collins, F. (2001). Academics' concerns about the "push for flexible delivery". Proceedings ASCILITE Conference, Melbourne, 377-386. http://www.ascilite.org.au/conferences/melbourne01/pdf/papers/lynchj.pdf [viewed 19 May 2002, verified 2 Sep 2002]

Collins, F. & Lynch, J. (2001). ICT education and the dissemination of new ideas: Channels, resources and risks. Proceedings AARE Conference, Fremantle. http://www.aare.edu.au/01pap/col01092.htm [verified 2 Sep 2002)

Lynch, J. & Collins, F. (2001). From the horse's mouth: Factors inhibiting and driving innovation in ICT education. Proceedings AARE Conference, Fremantle. http://www.aare.edu.au/01pap/lyn01093.htm [viewed 19 May 2002, verified 2 Sep 2002]

Author: Dr Juhani E. Tuovinen, Charles Sturt University, Wagga Wagga, NSW. Email: jtuovinen@csu.edu.au

Please cite as: Tuovinen, J. E. (2002). The National ICT Education Enquiry 2001. In S. McNamara and E. Stacey (Eds), Untangling the Web: Establishing Learning Links. Proceedings ASET Conference 2002. Melbourne, 7-10 July. http://www.aset.org.au/confs/2002/tuovinen.html

[ ASET ] [ Proceedings Contents ]
This URL: http://www.aset.org.au/confs/2002/tuovinen.html
Created 2 Sep 2002. Last revision: 2 Sep 2002.
© Australian Society for Educational Technology