Changing student use and perceptions of learning technologies, 2002-2004

The experiences of 504 students were explored by survey in their first year and their third year of study at university. Compared with first year, student engagement in paid work increased significantly and the number of days spent on campus decreased significantly by third year, though the overall number of hours spent attending classes did not change significantly. There were no significant differences in student levels of satisfaction with teaching or their university experience overall between first year and third year. Student use of information and communication technologies and their perceived usefulness of those technologies in supporting learning increased significantly over the two year period. Gender and age differences were apparent in the use of technologies. Frequency of use of learning technologies correlated with perceived usefulness of the technology. However there was a pattern of volatility in student use of technology over time in which early adopters later abandoned the technology.

Introduction

The effectiveness of learning technologies in improving student learning outcomes remains an area of continuing debate. In terms of the use of computer assisted learning, a review of 355 studies showed no significant difference in the effect of CAL in enhancing learning (Russell, 1999), whilst others have reported enhancement of learning under specific conditions (Andrew & Isaacs, 1995). In terms of student development, the use of computers has been correlated with self-reported gains in independent learning and problem solving (Kuh & Vesper, 2001), cognitive development and critical thinking (Flowers, Pascarella and Pierson, 2000). The use of learning technologies may also have some adverse consequences (Upcraft, Terenzini & Kruger, 1999) and the impact of learning technologies does not appear to be consistent for all students. Flowers, Pascarella and Pierson (2000) reported that cognitive effects of the use of computers were not uniform across the student body or institutions - higher performing students appeared to gain most. Student attitudes and perceptions may also play a significant role. Differences in access to the technologies, comfort and competence in use, preferred learning style and lifestyle or work patterns can all affect student use and benefit from learning with technology (Schrum & Hong, 2002).

First year at university is when most students will encounter their greatest exposure to learning technologies, although some school leavers may have used a LMS in their studies. The level of academic and social engagement in first year is the key to many students' experiences of, and success, in higher education (Cook & Leckey, 1999; McInnis, 2001). It is a major period of transition for many - matching expectations with reality and developing the necessary skills to become a self-directed independent learner in a technology rich learning environment. Many students entering university are uncertain what studying in a flexible learning environment involves for them as learners (Taylor, 2000). This is supported by data from a 1994 and 1995 snapshot study of the Australian first year university experience that revealed although many courses have learning technology support, up to a third of all students had not used the technologies (McInnis, James & Hartley, 2000). The purpose of this longitudinal study was to explore of a group of students' changing experiences and behaviours on campus and with learning technologies, in their first year and again in their third year at a large multi-campus metropolitan university. The nature and frequency of use of various learning technologies was examined together with self-reported usefulness of those technologies in supporting learning. Finally, student perceptions of quality and their overall experience at university were also explored.

Methodology

Results

Campus	N	%	Country of Birth	N	%
Nathan	185	36	Australian born	404	80
Gold Coast	167	33	Overseas born	98	20
Mt Gravatt	66	13	Age Groups
Logan	57	11	< 20	290	57
South Bank	29	6	20-25 years old	123	24
			>25 years old	82	19
Gender: Female	338	67	First generation student	292	58
Primary Carer	84	17	Primary Income Earner	100	19

A range of survey questions sought to evaluate students' time commitment to their university studies. In 2002, the majority of Griffith first-year and third-year students visited their home campus 3-5 days per week. Students aged over 25 years were less likely than younger students to visit the campus on more than 4 days per week. First year students spent less than 4days on campus (on average) and this decreased significantly (chi =10.65, p<0.001) as they progressed into the third year of their studies.

Commitment to the university was also examined in a series of questions asking about time spent on university studies, family and work related commitments (Table 2). The pattern of data from 2002 and 2004 were very similar. Time for preparation for classes did not change significantly, nor did the number of hours of class attendance decrease significantly from first year to third year. Students spent significantly more time engaged in paid employment as they progressed from first year to third year (chi =50.685, p<0.001). There were no significant differences in the proportions of students aged under 25 years and students aged over 25 years who were engaged in paid employment in 2002 or 2004.

Item	Year	Hours
		0	0-5	6-10	11-15	16-20	21-25	26+
Hours of preparation for class	2002 2004	0.5 2	26 23	24 20	20 20	12 14	8 12	8 8
Hours attending classes	2002 2004	0 8	3 18	10 31	40 21	26 14	14 7	4 0
Hours in paid employment* McInnis et al, 2000	2002 2004	37 21 40	9 7 9	12 15 19	16 19 15	12 16 10	8 11 3	4 10 4
After McInnis et al (2000), Table 4.2, 4.5 and 4.6.   * p<0.001

Section 1: Learning activities and experiences

	Year	Disagree	Neutral	Agree
I enjoy the intellectual challenge of my courses. McInnis et al, 2000	2002 2004	3 5 12	19 10 27	76 85 61
I enjoy the theoretical content of my courses. McInnis et al, 2000	2002 2004	8 12 24	28 16 36	64 72 40
Lectures and presentations stimulate my interest in the course. McInnis et al, 2000	2002 2004	13 19 20	34 24 35	53 57 46
I have found most of my courses really interesting. McInnis et al, 2000	2002 2004	20	29	50
Lectures are a valuable learning experience for me. McInnis et al, 2000	2002 2004	10 18 15	24 19 27	64 63 58
I get a lot of satisfaction from studying. McInnis et al, 2000	2002 2004	12 16 25	35 26 35	57 58 40

In terms of academic application, nearly half of the first-year and third-year students reported difficulty with motivation for study, even though many worked consistently throughout the semester and expressed a very strong desire to do well (Table 4). In third year, about 10% of students reported that they sought advice from teaching staff more often than they did in first year, but also they worked less consistently throughout the semester. Overall, there was a moderate correlation between academic application in 2002 and in 2004 (r=0.413, p<0.01), but no significant difference in the level of academic application of students in first year compared with third year.

Students perceptions of teaching
Student perceptions of teaching were evaluated using questions drawn from the national First-Year survey (Table 5, McInnis et al, 2000). The majority of first year students in 2002 reported that staff were enthusiastic and tried to make courses interesting, were approachable and available, good at explaining things and provided helpful feedback on student progress throughout the semester (Table 5). Students reported strong agreement with the statement about helpful feedback and taking an interest in student progress. There was a significant correlation between students perceptions of teaching in 2002 and in 2004 (r=0.317, p<0.01).

Section 2: Use and value of learning technologies

Use of learning technologies
Students used communication and learning technologies with increasing frequency over the last three years (Table 7). The frequency of use of email to contact staff and fellow students increased significantly (chi=20.56, p<0.001) (chi=53.15, p<0.001). This increase was largely due to higher usage by students aged under 25 years. It is interesting to note that about 15% of students, across all age groups, still did not use email to contact other students or staff at the university. The use of discussion forums (chi=100.76, p<0.001) and course websites accessed from home (chi=157.81, p<0.001) or accessed from work (chi=169.40, p<0.001) also increased significantly. In 2002, about one third of first-year students reported rarely or never using SMS text messaging to contact other students or email to contact staff and fellow students. By 2004 this proportion had almost halved , though 21% of students who rarely/never used text messaging to contact peers in 2002 continued to rarely or never use it in 2004 (Table 7). There was no correlation between overall frequency of CIT use and perceptions of teaching or overall satisfaction with university experience.

	Year	Disagree	Neutral	Agree
Staff try hard to make the courses interesting. McInnis et al, 2000	2002 2004	8 10 17	21 17 34	71 73 50
The teaching staff are good at explaining things. McInnis et al, 2000	2002 2004	8 11 17	32 22 35	60 67 48
Teaching staff are enthusiastic about the courses they teach. McInnis et al, 2000	2002 2004	7 9 12	24 19 32	69 72 56
Most of the teaching staff are approachable. McInnis et al, 2000	2002 2004	4 6 12	14 9 26	82 85 62
Staff are usually available to discuss my work. McInnis et al, 2000	2002 2004	11 13 25	31 23 37	58 64 38
Teaching staff usually give helpful feedback on my progress through the course. McInnis et al, 2000	2002 2004	23 23 40	37 29 34	41 47 25
Teaching staff make a real effort to anticipate difficulties that I may be having with my studies.	2002 2004	16 18	43 43	39 41
Teaching staff make a real effort to respond to difficulties that I may be having with my studies.	2002 2004	12 11	37 36	51 52
Most teaching staff in my courses take an interest in my progress during the semester. McInnis et al, 2000	2002 2004	27 28 47	39 35 32	33 36 21
Interaction with the teaching staff is an important part of my learning in this course.	2002 2004	11 11	28 22	61 67
Table 5.1, McInnis et al (2000).

Flexible learning means ...	2002	2004
More choice over what I learn	4	7
Learning at places of most convenience	5	10
Less obligation to attend classes	7	7
Group work	9	4
Access to a range of learning materials and resources	10	13
Taking more responsibility for my own learning	11	18
Access to online materials	12	18
Learning at my own pace	12	10
Learning at time of convenience	12	7
Less staff contact	16	5

The frequency of use of technologies differed by gender and across age groups. In 2004, older students were significantly more likely than those aged under 25 years to email academic staff members about study matters (chi =14.9, p<0.01). Younger students were more prolific users of text messaging (chi =39.81, p<0.001), with males being the greatest users of text messaging for peer communication (chi =8.40, p<0.01) and communication about study matters (chi =22.4, p<0.001). There were no gender or age differences in use of discussion forums or accessing of online information from work or home. Blogs were read or written on a regular basis by 75% of students in 2004. This behaviour was equally spread across broad age groups, however women read and wrote web logs significantly more frequently than men (chi =8.1, p<0.01).

Please indicate how frequently you do the following:		Rarely/ Never	Occasionally, Often, V Often
Use email to contact fellow students about study matters?	2002* 2004	37 16	63 84
Use email to communicate with staff members about study matters?	2002* 2004	31 17	69 83
Use SMS text messaging to contact other students?	2002* 2004	34 20	76 80
Use discussion forums on course websites to communicate with other students and staff? McInnis et al, 2000	2002* 2004	45 14 91	55 86 30
Access course websites, library or university information from on-campus? McInnis et al, 2000	2002* 2004	34 8 67	66 92 33
Access course websites, library or university information from home or work? McInnis et al, 2000	2002 2004	37 10	63 90 60
After McInnis et al (2000), similar data from Table 3.7 and 3.9.   * p<0.001

There were consistent patterns in students' changing use of technologies. These patterns were observed in frequency of student use of email to contact peers, use of email to contact academic staff about study matters and use of text messaging to contact peers. It was also evident in students' perceived usefulness of email contact with staff and peers for study matters, and learning benefits derived from participation in discussion forums. The pattern is illustrated in one example regarding student use of email to contact academic staff about study matters (Table 8). It can be seen that of the early adopters of the technology who used email frequently to contact academic staff in 2002, 50% now use email rarely or never to contact academic staff in 2004. The overall significant increase in use of email to contact academic staff from 2002 to 2004 is due to 15% of students who occasionally used email using it more frequently to contact academic staff in 2004, and 53% of students who rarely used email then, now using it occasionally or frequently to contact academic staff in 2004. Nearly half of the students who rarely or never used email to contact academic staff in 2002 continued not to use email to contact academic staff in 2004.

2004	2002
	Very Often/Often used In 2002 (n=114)	Occasionally used In 2002 (n=231)	Rarely/Never used In 2002 (n=159)
Very Often/Often	18%	15%	18%
Occasionally	32%	39%	34%
Rarely/Never	50%	46%	47%
	100%	100%	100%

Consonant with questions about the frequency of use of communication technologies for learning, were questions about the usefulness of those technologies in supporting student learning (Table 9). There was a strong correlation between frequency of use of technologies and their perceived benefit for learning (r=0.566, p<0.01). There was no correlation with perceptions of teaching, satisfaction or GPA. In 2002 around half of the first year students indicated that email contact with staff and other students was useful for their learning. Discussion forums were not used by nearly 30% of first-year students in 2002 (they chose not applicable), but of those students who did use discussion forums, one third found them to be useful for their learning. From 2002 to 2004, there was a significant increase in the number of students who reported that email contact with other students (chi =70.69, p<0.001) and staff (chi =9.95, p<0.001) was useful for their study. There was also a significant increase (chi =75.29, p<0.001) in perceptions of the usefulness of discussion forums to support learning over the same period of time.

Accessing course websites was easy for the majority of students in 2002 and this continued in 2004. Notably some 8% of students continued to report difficulty in accessing web-based materials in 2004: only one quarter of these people reported such difficulties in 2002. The great majority of students reported that accessing course materials from on-campus or at home/work was useful for their learning and in 2004 there was no significant change in their perception of the usefulness of those resources in supporting their learning. In general cross-tabulations reveal that the majority of students (60-70%) had not changed their opinions during the period of time between surveys. Responses were consistent across some items. For example, the students who found the use of online materials to be motivating in 2002 were largely drawn from the same pool of students that reported that access to course materials from on-campus and home to be useful for their learning. The majority of students who reported that accessing online materials was useful for their learning in 2002 reported similarly in 2004.

	Year	Disagree	Neutral	Agree
Email communication with other students was important for my study and learning.	2002* 2004	28 12	27 17	39 71
Email contact with teaching staff was important for my study and learning.	2002* 2004	14 13	29 16	52 71
Participation in online discussion forums with staff and students was important for my study and learning.	2002* 2004	27 30	28 22	17 27
The course web sites play an essential role in helping me learn.	2002 2004	8 12	21 14	69 74
Accessing web-based resources and information was easy for me.	2002 2004	8 8	11 9	71 83
Accessing web-based resources and course information from home was useful for my study and learning.	2002* 2004	4 11	8 9	80 82
Accessing web-resources and course information on campus was useful for my study and learning.	2002 2004	6 9	20 13	70 78
The use of web-based resources increased my motivation to study and learn.	2002 2004	16 19	32 31	50 50
I prefer using printed materials rather than on-screen materials when I have to think carefully about what I am reading.	2002 2004	5 9	15 13	78 78
I learn better using computer-based materials (Internet, CD-ROM, simulations) rather than printed materials.	2002 2004	42 44	40 36	18 20
I prefer to learn from information on the Internet rather than textbooks or library books.	2002 2004	38 35	35 34	24 31
I have become more critical of information I find on the Internet.	2002 2004	5 4	23 11	70 85
Absent percentages refer to N/A choice. After Tables 3.7, 3.9 and 3.10, McInnis et al (2000).  * p<0.001

The overwhelming majority (80%) of first-year students in 2002 indicated that they preferred to use printed materials when thinking carefully about the materials was necessary (Table 9). The majority of students (60%) did not change their preference for printed materials from 2002 to 2004. Of those students who were unsure of their preference in 2002 (n=75) or who preferred screen materials (n=21), 52 of 75 and 11 of 21 reported in 2004 a preference for print over screen-based materials. Only two students continued to prefer screen based information over print materials for the purposes of close study. The student population in 2002 was split as to whether they preferred to learn using technology-enhanced resources or printed materials and this situation had not changed significantly in 2004. A significantly greater number of students (chi =10.51, p<0.001) reported an awareness for increased critical evaluation of Internet materials in 2004 compared with 2002.

Student satisfaction
Students were asked about their overall satisfaction and perceived benefit of their university studies over the last two years (Table 10). There was a moderate correlation between students' level of satisfaction and perception of worth of their university experiences in both 2002 (r=0.370, p<0.001) and 2004 (r=0.495, p<0.01). There were no significant differences in each student's reported levels of satisfaction or perceived worth of their university experiences in first year compared with third year, by age, gender, or country of birth.

	Year	Disagree	Uncertain	Agreement
Overall, I am satisfied with my university experience so far.	2002 2004	5 8	23 17	77 75
Overall, my university experiences have been worthwhile.	2002 2004	2 6	11 11	87 84

Discussion

The first year academic experience of students in this study compares well with the aggregated national data of first-year attendance reported by McInnis et al (2000). Our first year students tended to have fewer contact hours, spend less time on campus and work slightly longer hours than the national average. Our first-year students. reported greater enjoyment of the intellectual challenge and the theoretical content of their courses. Whilst they reported greater learning value derived from attending lectures, the level of stimulation from those lectures was lower than the national experience. Overall first-year students held generally more favourable perceptions of teaching and greater satisfaction with their university experience than those reported as the aggregated national first-year student data.

The adoption of an "on-campus and on-line" approach to flexible learning appears to have resulted in mixed messages for first year students. This may be related to variable experiences in difference courses, however many students saw flexible learning as providing access to a range of online resources and less than expected contact with academic staff. Significant numbers of students reported inadequate feedback from, and limited availability of, academic staff. It is very likely that an unintentional over-emphasis on the use of IT occurred at the expense of structured class time, a situation typical of flexible learning environments (Nunan, 1996). These data offer support for Taylor's (2000) argument that more should be done to inform students of the aims of flexible learning and to prepare them better to make the transition to more independent modes of learning.

In 2002, first year students at our institution showed greater frequency of use of learning technologies than that reported nationally in 1999 (McInnis et al, 2000). Email was used regularly by about three fifths of student at our institution, which was slightly higher than shown in the national data. There was also greater use of online course materials accessed from home or on-campus and higher participation in discussion forums. The latter is attributed to the adoption of a learning management system (LMS) at our institution and such LMSs were not well established nationally at the time of McInnis' survey in 1999. Notably, about 40% of students in this 2002 study, in a larger first-year study (Zimitat, 2003), and in that of McInnis et al (2000) rarely used email or accessed course information.

Overall changes in use of learning technologies by third year

The use of learning technologies has been positively associated with other forms of student engagement (Kuh & Hu, 2001). In this study there was a marginal correlation (r<0.25) between use of technologies, perceived usefulness of technologies and satisfaction, GPA and perceptions of teaching. Aggregation of data from several years may provide sufficient statistical power to further test these relationships.

Individual changes in use of learning technologies

Further work

References

Bell, M., Bush, D. Nicholson, P., O'Brien, D. & Tran, T. (2002). Universities Online. Occasional Paper Series, Higher Education Group, DEST. http://www.dest.gov.au/highered/occpaper/02a/02_a.pdf

Cook, A. & J. Leckey. (1999). Do expectations meet reality? A survey of changes in first year student opinion. Journal of Further and Higher Education, 23, 151-157.

Flowers, L., Pascarella, E.T. & Pierson, C.T. (2000). Information technology use and cognitive outcomes in the first year of college. Journal of Higher Education, 71, 637-667.

Kuh, G. & N. Vesper. (2001). Do computers enhance or detract from student learning? Research in Higher Education, 41, 87-102.

Kuh, G. & S. Hu. (2001). The relationship between computer and information technology use, selected learning and personal development outcomes and other college experiences. Journal of College Student Development, 41, 217-232.

McInnis, C. (2001). Researching the first-year experience: Where to from here? Higher Education Research & Development, 20, 105-114.

McInnis, C., James, R. & Hartley, R. (2000). Trends in the first-year experience in Australian universities. Evaluations and Investigations Programme, Higher Education Division, DETYA. Canberra: AGPS. http://www.detya.gov.au/archive/highered/eippubs/eip00_6/fye.pdf

Merisotis, J. (1999). The "What's the difference debate?" debate. Academe, 85, 47-51.

Nunan, T. (1996). Flexible delivery - What is it and why is it a part of current educational debate? Different Approaches: Theory and Practice in Higher Education. Proceedings HERDSA Conference 1996. Perth, Western Australia, 8-12 July. http://www.herdsa.org.au/confs/1996/nunan.html

Owston, R.D. (1997). The World Wide Web: A technology to enhance teaching and learning. Educational Researcher, 26, 27-33.

Lenhart, A. (2003). The ever shifting Internet population. [11 Jul 2004, verified 29 Oct 2004] http://www.pewinternet.org/pdfs/PIP_Shifting_Net_Pop_Report.pdf

Russell, T.L. (1999). The no significant difference phenomenon. International Distance Education Certification Center. [viewed 12 Jan 2003 at teleeducation.nb.ca/nosignificantdifference/, verified 29 Oct 2004 at http://www.nosignificantdifference.org/]

Schrum, L. & S. Hong. (2002). From the field: characteristics of successful tertiary online students and strategies of experienced online educators. Education and Information Technologies, 7, 5-16.

Song, L., Singleton, E.S., Hill, J. &. Koh, M.H. (2004). Improving online learning: Student perceptions of useful and challenging characteristics. The Internet and Higher Education, 7, 59-70.

Taylor, P.G. (2000). Changing expectations: Preparing students for flexible learning. International Journal for Academic Development, 5, 107-115.

Upcraft, M. L., Terenzini, P. T. & Kruger, K. (1999). Looking beyond the horizon: Trends shaping student affairs -technology. In C. Johnson and H. Cheatham (Eds.), Higher education trends for the next century: A research agenda for student success (pp. 30-35). Washington, DC: American College Personnel Association.

Zemsky, R. & Massy, W.F. (2004). Thwarted innovation. What happened to e-learning and why. The University of Pennsylvania, PN: The Learning Alliance. http://www.irhe.upenn.edu/Docs/Jun2004/ThwartedInnovation.pdf

Zimitat, C. (2003). The impact of full-time employment and family commitments on the first year experience of full-time students. Paper presented at the 7th Annual Pacific Rim First Year Experience Conference, QUT.

Author: Craig Zimitat, Griffith Institute for Higher Education, Griffith University, Brisbane, Australia. c.zimitat@griffith.edu.au Please cite as: Zimitat , C. (2004). Changing student use and perceptions of learning technologies, 2002-2004. In R. Atkinson, C. McBeath, D. Jonas-Dwyer & R. Phillips (Eds), Beyond the comfort zone: Proceedings of the 21st ASCILITE Conference (pp. 984-993). Perth, 5-8 December. http://www.ascilite.org.au/conferences/perth04/procs/zimitat.html

© 2004 Craig Zimitat
The author assigns to ASCILITE and educational non-profit institutions a non-exclusive licence to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. The author also grants a non-exclusive licence to ASCILITE to publish this document on the ASCILITE web site (including any mirror or archival sites that may be developed) and in printed form within the ASCILITE 2004 Conference Proceedings. Any other usage is prohibited without the express permission of the author.