Ken Ryba and Tom McIvor

Massey University

Maha Shakir

Zayed University

Di Paez

University of the Sunshine Coast

Abstract

This study examined continuous automated speech recognition in the university lecture theatre. The participants were both native speakers of English (L1) and English as a second language students (L2) enrolled in an information systems course (Total N=160). After an initial training period, an L2 lecturer in information systems delivered three 2-hour lectures over a three-week period to the participants and other students. Student self reports indicated that there were a number of perceived benefits associated with the use of continuous automated speech recognition. Compared with L1 students, a significantly greater number of L2 students and special needs students reported that the system had potential as an instructional support mechanism. However, a greater accuracy in the system’s recognition of lecture text vocabulary needs to be achieved. The implications are that lecturers need an extensive training period before delivering lectures using continuous automated speech recognition.

Introduction

The Liberated Learning Project (LLP) is an applied research project, which looks at two main questions:

1. Can continuous speech recognition (CSR) technology be successful at displaying speech-to-text in university classrooms to provide universal access to lecture material for students from diverse backgrounds?

2. Can CSR be a successful alternative to traditional styles of lecturing? (Bain et al. 2002:192).

The LLP was devised and initiated by Saint Mary’s University (Halifax, Canada) in 1998 in association with IBM. Subsequently, an international consortium was developed with the aim of further refining and researching the LLP (Bain et al. 2002:192). Members of the consortium currently include: Alexander Graham Bell Institute; University College Cape Breton, Canada; Trent University, Canada; Cambrian College, Canada; Purdue University, USA; California State University Northridge, USA; Messiah College, USA; Massachusetts Institute of Technology, USA, University of the Sunshine Coast, Australia; Australian National University, Australia; Central TAFE College, Australia; Murdoch University, Australia; and, Massey University, New Zealand. Working in association with the University of the Sunshine Coast, Massey University is the first New Zealand tertiary education institution to trial the LLP technology in New Zealand.

The aim of the Liberated Learning concept is to provide universal access to lecture information for students from diverse backgrounds.

It does this by incorporating CSR as a natural extension of the instructional process within the university classroom. An illustration of the LLP system and how it works in practice is displayed in Figure 1 below.

Early on, the Liberated Learning team realized that commercially available speech recognition software (ViaVoice™) was not conducive to use in the classroom environment. As such, IBM in collaboration with the Liberated Learning team created the first classroom speech recognition technology that would successfully digitize a spoken lecture and display output in readable form (Viascribe™). Viascribe software includes the need to use no punctuation; an algorithm utilizing naturally occurring pauses in speech causes the displayed text to move to a new line, creating automatic readability of text. After the lecture has been delivered, the lecture is edited, punctuation is inserted, recognition errors are corrected and redundancies removed. The software can synchronize text and speech data to create bi-modal multimedia lecture notes, accessible in multiple formats via the internet - text, audio, or synchronous text and audio transcriptions.

After training in the use of automated speech recognition software (voice-to-text) (IBM’s ViaVoice and Viascribe), faculty members, wearing cordless microphones use CSR in their lecture theatres. Their spoken lectures are digitized and simultaneously translated into text using the Viascribe software, then displayed on a large screen in front of the lecture theatre so students can both see and hear the lecture. (Paez 2002, p. 920)

Figure 1: Stand-alone Liberated Learning System

The text is simultaneously displayed via projector in real time-students can simultaneously hear and see the lecture as it is delivered. In this way, it can reduce cognitive load for students by increasing short-term memory resources. Therefore, the LLP system can potentially benefit all students, especially those with special learning needs. This includes students who:

- are deaf or hard of hearing;

- have medical health and physical conditions that prevent their full participation in lectures;

- have learning difficulties;

- use the instantaneous display of the lecture as a reference check for their notes.

LLP also aims to provide students from non-English speaking backgrounds with a tool to give them greater access to lectures, thereby “liberating” their learning options (Leitch & MacMillan, 2001). CSR technology provides a real-time digital display of spoken lectures/lessons and from this, on-line transcripts of those lectures/lessons. The on-line transcripts then become available for students to use through access to WebC™. A finding in Heller (2004:17) is that 65% of students who have access to Liberated Learning use these on-line transcripts. Thus the aim of liberated learning is to further CSR supported lecture comprehension among students with disabilities and among L2 English as a foreign language students in the lecture theater (Leitch & MacMillan 2003:9).

Listening to academic lectures can be challenging for many L2 students and they often fail to understand the main points of lectures (Jung 2003:562). One reason for this failure is that lectures place a too heavy cognitive load on L2 students in terms of auditory processing (Thompson 2003:5). Therefore, additional processing support in the form of simultaneous visual text may give L2 students a support system with which to compensate for any deficiencies in listening comprehension ( Leitch & MacMillan 2003:10). Liberated Learning can provide listening support through scaffolded instruction utilising synchronous text displays and asynchronous online streaming of spoken lecture speech.

To this end, findings by Leitch and MacMillan (2003) report the following positive uses of the Liberated Learning speech display:

a)compensating for lecture information that was missed

b)comparing on-screen text to the spoken language of the lecture

c)supporting students with the pace of the lecture (Wilkes et al. 2003, p. 8).

LLP is in its infancy and there are three very important challenges for LLP to meet before it can become a successful alternative to traditional methods of lecturing:

1. It must perfect the accuracy of the electronic text

2. It must generate an increase in quality and quantity of note taking

3. It must improve the readability of displayed text (Bain et al.2002, p.194).
   

This investigation looks at these challenges in the context of an L1 (native speaker) and L2 lecture learning context.

Method

Research Questions

The present study was designed to explore the practical application of continuous automated speech recognition. The purpose of the study was to trial the use of automated speech recognition as a means of improving access to learning materials for students with disabilities and English Language support needs.

Specific attention was given to:

• an examination of the viability of using continuous text-to-speech conversion in the university classroom

• an investigation into students’ perceptions and experiences with using LLP technology to scaffold their instruction

• a comparison of learning outcomes, study preferences, and class experiences of L1 and L2 students who elect to use the continuous text-to-speech conversion.

The specific questions guiding this study included:

1) What are students’ perceptions of using the speech-text display and the streaming?

2) To what extent do students make use of the facilities?

3) What do students consider to be the main advantages of using LL?

4) What are the limitations and problems reported by students?

Participants

Participants in this study were students enrolled in an introductory level information systems course (N=160) of whom approximately 50% were L2 students. In total, there 81 L2 students and 79 L1 students who agreed to participate in the pilot project. Lecture 2 was attended by 139 students (L1=75) (L2=64). Lecture 3 was attended by 136 students (L1=71, L2=65). Lecture 4 was attended by 119 students (L1=64, L2=55). The three lectures were well attended both by L1 and L2 students which makes the results robust.

Informed Consent Procedure

At the first lecture of the term, the researchers presented information on the project and invited students to participate. All students were given an information sheet and consent form. Students who elected to participate in the project left their signed consent form on a chair next to the door as they exited the lecture theatre. It was stressed that participation in the project was not a course requirement and would have no effect on their grade. An advantage of the design is that all students could make full use of the LLP system and online streaming facilities whether or not they agreed to be included in the project. In order to facilitate a comparison of L1 and L2 students, participants were asked to print and sign their name on a consent form and on a questionnaire. Only the researchers had access to the data. The course lecturer had no access whatsoever to email exchanges, online discussion forums or the class survey. The procedure for conducting this study was formally approved by the Massey University Human Ethics Committee.

Voice Recognition Training

The lecturer underwent training to develop a voice profile for the continuous speech recognition system. This involved the lecturer with the aid of a computer technician inputting dialogue and vocabulary into a ViaVoice™ speech recognition system. This training is intensive and requires patience on the part of the lecturer to achieve a high level of accuracy.

Description of the LLP System Setup

A stand alone system was used for the pilot project. This comprised:

1. Laptop computer

2. wireless microphone set

3.Viavoice 10™ – local voice profile

4. Viascribe™ display interface for automatically transcribing speech into text.

5. Text output to single in-class display via data projector

6. File storage on local hard disk

7. File transfer and editing

8. Lecture files uploaded to internal network

Prior to the commencement of each lecture, the system was set up to complement the existing instructional facilities (i.e. projector, microphone, PowerPoint™)

The lecturer used a headset with a wireless microphone attached to a lap computer. The lecturer’s voice profile was then loaded so that the speech recognition could first be tested for voice quality.

The lecturer speaks and the acoustic information is translated via Viascribe™ into electronic text that is displayed by a beam projector onto a screen. Viascribe™ instantly creates a series of accessible multimedia files that can be easily published through learning portals such as WebCT™.

Post lecture, the electronic files were sent via FTP to the University of the Sunshine Coast for editing. The corrected files were then returned to Massey University where they were placed on a website accessible via the class WebCT™ site.

Within the class WebCT™ site, students could select either the first or second part of each lecture that was recorded. They could then view the lecture text and hear the audio presentation simultaneously while viewing PowerPoint™ slides.

Measures

Email exchanges: The researcher sent an email after lecture 2 to each student inviting them to comment on the following points:

1. What are your initial impressions of speech recognition?

2. If the accuracy in the on-screen text could be improved, do you think this could help your learning?

3. What do you think the problems are with this technology?

4. Do you find the visual display of simultaneous text distracting?
   

WebCT discussion forum: all students had access to a class WebCT™ site to support their coursework. A discussion forum was set up within the class site concerning the use of the LLP technology. Students who had agreed to participate in the project were invited to make comments on their own experiences and thoughts about using the text-to-speech conversion and the video streaming. Students were reminded and encouraged at each lecture to post messages on the discussion forum.

Class Survey: The class survey was administered at the beginning of lecture 6 - two weeks after the last LLP trial lecture. This provided increased time for students to try accessing and using the streamed lectures via WebCT™ for revision.

The survey asked students to:

1. indicate which lectures they attended where the LLP system was used

2. state how much of the speech-text display they made use of

3. rate the extent to which they accessed and used the streamed audio and text files

4. indicate their agreement or not with 9 statements concerning the perceived effectiveness of the LL Project

5. describe perceived advantages, problems, and suggested improvements

Results

Email Responses from Students

A total of 10 email responses were received. The majority of these were positive indicating that the potential of LLP was recognised by both L1 and L2 students. The main themes in the responses were the use of compensatory strategies, the potential of LLP to distract from lectures, and the accuracy of the system. Following are some specific replies to the questions that were provided to the students.

An L1 student reported the following with regard to strategies:

”This has great potential to be standard in lecture theatres Massey wide. I found myself glancing at it now and then to try and pick up a word or two. Excellent idea. Keep up the good work!”

The following is a response from an L1 special needs student:

“My initial impressions of speech recognition is very positive. I am deaf in one ear so it can help people like me who may find it difficult to hear or understand what the lecturer is saying.”

The student recognizes the potential of LLP as an educational school, but with the qualification that improvements should be made to it:

“Yes I do think this could help my learning if the accuracy was improved.”

Similarly, L2 students used the system as a strategic tool with which to support any deficiencies in their listening to lecture skills as evidenced in the following response:

“The speech recognition is a fantastic idea to aid non-native students (like me!) I’ve been studying in English for a couple of years now, but since English is not my first language, I still sometimes have problems with my listening skill.”

The following response from this same L2 student also contains a call for improvement in the LLP:

“If the accuracy of the system could be improved, I would totally support the use of it at University; and I’m sure other non-native speakers would find the system helpful too.”

L1 students also called for improvements in LLP as the following example indicates:

“I have no problems listening to the lecturer with the visual display of simultaneous text, however, occasionally, I will look at the on-screen text and laugh, which disrupts me and also, disrupts others students who sit around me.”

L1 students complained about the potential of LLP to distract during lectures. The following email response is an example of how LLP can be distracting for L1 students:

“At this stage, the system has not yet achieved the right accuracy level, so I find it quite distracting at times. When I’m in the lecture, I often try not to look at the on-screen text as it’s very amusing and the sentences don’t often make sense; I might say that what appears on the screen are just lists of words rather than full, whole sentences.”

The above response is also critical of the text display format, but this was a rare criticism. The following response is critical of the way LLP can distract from the lecturer:

“Shifts the focus from the lecturer. And by watching the lecturer you can actually learn a lot about public speaking skills and things like that so the lecture can become more impersonal.”

This student goes on to make a prediction about the future of live lectures based on the availability of streamed voice files:

“Also – students won’t go to lectures they will wait for the file to be posted and learn that way which defeats the purpose of holding lectures.”

The same student complained about the colour of the background in the display:

“I have to try not to look at it all the time. The blue background colour hurts my eyes also.”

Finally, the following positive comment suggests the existence of a social responsibility and empathy amongst L1 students for any deficiencies in L2 listening to lecture skills:

“It is an interesting concept. Good to see the university trying to assist students in new ways. Especially international students”

WebCT Discussion Forum

Approximately 30 postings from both L1 and L2 students were placed within the discussion forum. There was no posting from students who identify as having special needs. The responses were generally positive but also identified some of the problems and issues that need to be addressed. These issues were mostly the strategic use of LLP, the system’s potential for distraction, and the system’s inaccuracy.

On the issue of compensatory strategy use, the following is typical of comments received from L1 students:

“I found myself glancing at the screen a couple of times to try and confirm a word I missed in the lecture. If this system becomes more accurate, I would not be surprised if the voice recognition system became standard in lecturing at every university. “

Postings usually contained both positive and negative comments such as the following:

“The only problems I have with it are its inaccuracy and initially it was almost a distraction. Great idea though. I feel proud to be part of a potential breakthrough. Keep up the hard work.”

L2 students echoed those L1 responses by describing how LLP can support L2 learning in the lecture theatre. The following comment indicates strategy use among non-native participants:

“English is my first language but sometimes I had troubles hearing what was said. It worked out well that the parts I misheard the voice recognition picked up fine.”

The following response indicates the extent of the support that LLP can offer L2 students to compensate for any deficiencies in listening comprehension:

“I think that is great!! Because English is our second language, it is not too bad when we can't hear the words from the teacher but we can see them on the screen. “

Students also commented on the potential of the system to distract students during lectures. The following comment was posted by an L1 student:

“Sitting in the middle left of the auditorium, I found the real time text distracted me from what the lecturer was saying as it was in my field of view. “

The next response which is from an L1 student blames this distraction on the system’s inaccuracy. The following contains suggestions for improvements to the LLP system:

“The idea is good in theory. The interpretations as they are now have no benefit at all because none of the sentences make any sense! Obviously it would work better if the speaker spoke clear English. It is a joke putting the interpretations up as they are now. Currently the program is very limited but will be good when it is refined. For now, it is a distraction in class, however, some interpretations are quite amusing!”

L2 students also identified inaccuracies in the displayed text as being responsible for any distractions. The following posting is from such an L2 student:

“On the other hand, it is quite hard to pay all the attention to both and some pronounce on the screen is not right, which is different to what the teacher said.”

The following posting from an L1 student sums up the general positive attitude towards LLP shown by both L1 and L2 students:

“I am pleased that Massey is developing innovative methods of lecturing. I found myself glancing at the screen a couple of times to try and confirm a word I missed in the lecture. If this system becomes more accurate, I would not be surprised if the voice recognition system became standard in lecturing at every university. The only problems I have with it are its inaccuracy and initially it was almost a distraction. Great idea though. I feel proud to be part of a potential breakthrough. Keep up the hard work.”

Survey Responses

How much use did students make of the speech-text display?

Table 1 shows the participants’ level of use. Over one third of all students (37%) reported that they did not use the display. These included 57% of L1 students, but only 18% of L2 students. This suggests that there may exist some need among L2 students for listening comprehension support in the lecture theatre. However, it also shows that more than two-fifths of L1 students felt a need to use the display.

Table 1:

Ratings of speech-text display use

Count

How much did students access and use the streamed speech-text and audio files?

Table 2 shows that only 73% of L1 students said that they did not need to access the files while 24.7% of L2 students reported that they did not need to access the display. Overall slightly less than half of students (48.4%) said that they did not need the display. This suggests that a majority of L2 learners need the support of LLP for lecture comprehension while a majority of L1 speakers do not.

Slightly less than half (46.9%) of L2 students made use of the “streamed” lectures for review purposes L2 students accounted for just. Finally while the majority of students could access the files, 18.9% of students overall said that they could not access the files because of technical problems.

This suggests that a number of L2 students feel that accessing the streamed lecture files is of use for review purposes. It would also indicate that a majority of L1 students neither want nor need to access the files. In addition, it would seem that almost one-fifth of students overall need support to access the files.

Table 2:

Access to Streamed Text and Audio Files

Count

Table 3 shows the frequency of use of the streamed files. Over two-thirds of L1 students (68.4%) reported not using the streamed files at all while over one third of L2 students (39.6%) said that they had not used them. Overall 51% of students said that they did not use them. While 60.2% of L2 students said that they had used the files, less than one-third of L1 students (31.3%) reported the same. Overall 48.9% of students used the streamed files. Table 3 indicates that while a majority of L1 participants ignored the streamed lecture files, a significant number felt that accessing the streamed lecture notes was beneficial for their study. It also might reveal the high need for listening comprehension support that L2 students require during lectures.

Table 3:

Ratings of the extent to which students used the streamed files

Count

How effective did students perceive the project to be?

Students were asked to rate their perceived effectiveness of the LL Project on a number of different factors. They were asked to rank their views on how effective LLP was by choosing from the following: strongly agree; agree; disagree; strongly disagree. Table 4 shows that slightly over two-thirds of all students consider that the LLP can be beneficial to students’ learning from lectures. It also indicates the significant differences in the perceptions of L1 and L2 students on these beneficial effects. Table 4 appears to confirm the need for listening comprehension support among L2 students. Over two-thirds of L2 students felt that the speech-to-text display aided their comprehension of the three lectures. In addition, over three quarters of L2 students felt that using the streamed lecture files increased their comprehension of the lectures. Conversely, 80.5% and 74% of L1 students disagreed that the display and the streamed files respectively supported their comprehension of the lectures.

The table also indicates significant differences in how students feel about the effect of LLP on their note taking. Less than one-quarter of L1 students said that the display helped them to take notes and almost three-quarters of them reported that the streaming did not aid their note taking. On the other hand, 63.3% and 72.7% of L2 students reported beneficial effects of the display and the streamed files respectively on note taking behaviour.

Table 4 shows that over half of students overall thought that LLP was successful. 58.8% of L2 students regarded the project as successful while 34.3% of L1 students thought so. This suggests that slightly more than two-fifths of L2 students found that LLP had failed to give them adequate support. Nevertheless, at least three quarters of L2 students said they would like to have LLP in their other classes and almost three quarters of L2 students thought that LLP was easy to use.

Over half of L1 students on the other hand would prefer not to have LLP in their other classes although over half of L1 students again found it easy to use. In addition over half of L1 students would recommend LLP to their friends suggesting possibly that many of them recognise the potential of LLP for students with language support needs.

Table 4:

Perceptions of the Effectiveness of the LLP System

What did students think were the main advantages, problems and suggested improvements that were needed?

Table 5 presents a thematic analysis of comments given by students on the following:

1. What do you think are the main advantages of the Liberated Learning system?

2. What do you think are the biggest problems with using the Liberated Learning system?

3. Is there anything that you think should be improved? If yes, what should be improved and why?

A majority of students felt that accuracy is an important variable in determining the success and future of LLP. Over half of participants thought that a lack of accuracy had a negative impact on learning while slightly less than one third of participants want accuracy to be improved. A typical theme comment was the following:

“If the accuracy increases it will be possible to take more comprehensive notes if the student doesn’t understand what the lecturer is saying”.

Over one-quarter of participants felt that LLP is useful to them for revision purposes. In addition, the potential of LLP to support note taking ability was mentioned by a number of participants. In this regard, some students highlighted the strategy value of LLP by commenting on their ability to scan and read the words on the screen.

Other participants found that the display was interfering with learning and distracting. The lecturer used PowerPoint™ displays during the three lectures and students had to split their attention between the speech-to-text display, the lecturer and the PowerPoint™ display.

Overall the themes identified in Table 5 demonstrate some recognition of the value of LLP as a support for L2 students’ listening comprehension needs and also as a support for note taking ability.

Table 5:

Thematic Analysis of Advantages, Problems and Suggestions for Improvement

Discussion

This exploratory study provides some overall evidence that the LL Project has the potential to enhance student learning in a number of important ways. This was evident in the responses from students on the survey as well as their comments on email and the postings on the discussion forum. The perceived benefits include enhanced comprehension of lectures and improved note taking skills.

A notable feature of the project was that students felt involved in pioneering a new application aimed at creating better conditions for learning. Several students commended the project and expressed a desire to see the project extended because they considered that it would be beneficial to L2 students and students with learning needs….”I feel proud to be part of a potential breakthrough”

L2 students comprised one-half of the overall sample of participants. Comparison of L1 and L2 students showed that there were significant differences in how students perceived the benefits of LLP. L1 students while welcoming the use of this new technology felt mainly that while it could be of benefit to them, it would be more beneficial for L2 students. These L1 perceptions are in keeping with previous anecdotal research findings, which report benefits mainly for special needs students and L2 students. Many L1 students found the text display distracting as they had to split their attention between lecturer, screen and PowerPoint™ display.

Almost twice as many L2 students as L1 students reported that they had used the streamed speech files. Nearly three times as many L2 students as L1 students said that they needed to access the streamed files and twice as many L2 as L1 students reported using the files. Consequently, the benefits of LLP seemed to have been felt more keenly within the ranks of the L2 students than L1 students. More L2 students than L1 agreed that LLP was effective in supporting their learning. In fact, more L2 students than L1 agreed that LLP aided both their listening comprehension and note taking skills. In addition, more L2 students considered that LLP was successful and significantly, more L2 students than L1 agreed that they would like to have LLP in their other subject classes.

L2 students find lectures challenging and they have difficulty understanding the main and supporting points of lectures. They also have problems with the discourse structure of lectures, speech rate, and vocabulary amongst others. The majority of L2 participants recognized the potential of LLP to support their learning. According to Moreno & Mayer (2002,156) one finding of research into the testing of Dual Processing theory is that redundant information presented in two modes (text and narration) and containing similar words processed aurally and visually can support the recognition and learning of that input. Thus, in keeping with the finding of Moreno and Mayer (2002), L2 participants used strategies such as scanning the text display when they missed parts of the lectures. In this way, LLP can provide much needed support for L2 students in the lecture theater as L2 students process aural text with the help of simultaneously displayed on-screen text (Jones & Plass 2002, 548).

It was notable that some students mentioned the benefits of having speech recognition for words with which they had trouble hearing or were unfamiliar. Clearly, one of the benefits of LLP is that it can be taught to recognise technical words and speech patterns that are regularly mentioned and important for understanding the lecture content. This can assist students, especially those from other languages, to focus on the instructional material with more visual text scaffolding than would otherwise be possible in the conventional situation.

Quite a number (52) of students overall accessed the streamed files and found these useful for reviewing the lectures as indicated in the results. In projects of this kind, the streaming aspect is essential as an extension of the in-class speech-to-text conversion. This has perhaps the greatest potential to improve learning as it makes it possible for students to review the lecture in its entirety as through they were actually present. The addition of video would make it possible to capture all of the lecture and interactions in both visual and auditory form. The potential of LL for distance education is extremely significant and is now ready for further development by tertiary institutions who wish to deliver their programmes both nationally and globally.

It goes without saying that not all students will benefit from the LLP approach. The most effective use of the system would appear to be as an adjunct to the existing lecture approach and as a means of capturing the lecture material for later presentation or streaming over the internet. While many students in the present project said that they did not need the support of LL, it may be that they would benefit from the lecture review opportunities that it provides, especially in preparation for examinations. The practical consideration is that students should be encouraged to access and use the streamed lecture material for review along with other conventional approaches.

One of the issues with introducing this form of technology into the lecture theatre is the set-up time and the procedure, which can be somewhat distracting for students at the beginning of a class. To overcome this issue somewhat, the researchers provided information to students and had a dialogue with them about the project activities. This approach was helpful for creating a sense of participation amongst the class.

The project was originally scheduled for an entire University term (12 weeks) although due to the early departure of the participating lecturer overseas, it was essential to abbreviate the project. The three-week pilot was useful, however, for working out the systems and arrangements that were needed to support a larger project. This included: (1) working out how to record and then prepare files for editing; (2) how to make the files accessible so that they could be streamed in conjunction with PowerPoint™ presentations and other applications; (3) setting up a discussion forum on the class WebCT ™ site; and, (4) developing a survey to measure students attitudes and experiences.

Accuracy of the speech recognition was a significant limiting factor in the present study. This is evident in the fact that students in the survey commented on the need for improved accuracy. It is estimated that accuracy was at approximately the 85 percent level and that ideally this needs to be improved. It should be kept in mind, however, that the participating lecturer only had a limited speech training period of 2-3 hours and the fact that she was a second language speaker no doubt had an effect. Considering these factors, the accuracy level was quite reasonable and in most cases was acceptable to students in the class. There is definitely room for improvement of speech-to-text conversion accuracy and no doubt this can be achieved through a larger amount of training and improved acoustics and other technical factors.

Conclusion and Practical Recommendations

Experiences gained in this exploratory project indicate a number of areas in which improvements could be made and further research and development work undertaken:

1. improved speech recognition accuracy – this could be achieved through increased speech training within the classroom situation along with practice in using the system prior to the commencement of the academic term

2. improved setup procedure that is less distracting – the most effective approach would be to incorporate the LLP system into the technological setup of the lecture theatre or classroom. Alternatively, adequate time needs to be available prior to the class in order to prepare so that the LLP system is ready to commence at the outset of the lecture once a sound check has been carried out

3. more interactive features to improve communication with students – e.g. working as a class to identify significant points for discussion that are then read onto the speech text screen so that everyone can see the points raised. A more elaborate example would be a class consultation to develop a statement on how to solve a particular problem or issue. The statement would then be read and made available online

4. seating arrangements so that there is a specific area where people who wish to see the speech-text screen can chose to sit in that area. The screen is dominant and easy to see in this area and does not distract others who do not which to view the speech text.

5. more focused studies are needed to examine the outcomes of LLP on student note taking, comprehension, recall, and achievement

6. selective use of speech recognition is preferred. For example, certain sessions where material is largely visual in form or there are discussions may not be conducive to LLP whereas formal lectures especially introducing new concepts, vocabulary and technology may benefit especially from the use of speech recognition.

7. active participation of students in the LLP process could lead to a greater sense of involvement in class activities. Interactive methods could promote more active and meaningful participation in class.

8. dual and multiple processing theory would indicate that split attention problems need to be carefully managed. This could be accomplished by providing students with some advice on when and how to use the LLP system within the class. For example, L1 students may only use the system occasionally in-class to check spelling or pronunciation and refer to the streamed text afterward for a full review of the lecture.

In conclusion, this is a promising technological approach that has great potential for enhancing learning outcomes. There is a need, however, to match the technology more carefully with the pedagogical processes in order to ensure that better learning conditions are created. There are many issues and limitations that need to be resolved but current work within the consortium in partnership with the ICT industry will help to improve the effects. A key factor in favour of speech recognition in the way that it has been applied here is to provide universal access and support to a full range of students including those with special needs and for speakers whose first language is not English.

Acknowledgement

The authors are indebted to Stephen Ford, IT Consultant and Lorri O’Brien, IT Support Manager, Institute of Information and Mathematical Sciences, Massey University, Albany Campus who provided technical support for the project. Appreciated is also extended to Tony Richardson, Coordinator of 157.100 Introduction to Information Systems who facilitated this exploratory study within the class.

References

Bain, K., Basson, S.H, & Wald, M. (2002). Speech recognition in university classrooms: Liberated Learning Project. Proceedings of ASSETS 2002, Fifth International ACM SIGCAPH Conference on Assistive Technologies, Edinburgh, Scotland.

Heller, N. (2004). Messiah College in select group using new assistive technology: Liberated Learning boosts comprehension with the aid of real-time, closed-captioning of lectures. The Messiah College Bridge (summer). Grantham Pennsylvania: Messiah College.

Jones, L.C., & Plass, L.J. (2002). Supporting listening comprehension and vocabulary acquisition in French with multimedia annotations The Modern Language Journal 86 (iv), 546-561.

Jung, E.H. 2003 The role of discourse signaling cues in second language listening comprehension, The Modern Language Journal 87 (iv), 562-577.

Leitch, D., & MacMillan,T. (2003) Innovative technology and inclusion: Current issues and future directions for Liberated Learning research. Unpublished research paper, Saint Mary’s University, Nova Scotia.

Leitch, D., & MacMillan, T. (2001). Liberated learning Project: Improving access for persons with disabilities in Higher education using speech recognition technology: Year 2 report. Saint Mary’s University, Nova Scotia

Moreno, R., & Mayer, R.E. (2002). Verbal redundancy in multimedia learning: When reading helps listening. Journal of Educational Psychology 94 (1), 156-163.

Paez, D. (2002). Using Speech Recognition Technology in the Classroom: New ways of Learning. International Journal of Learning, 9, 911-923.

Paez, D., Wilkes, M., & Gurgone, M. (2004). Vocational Education and Training Engaging with Emerging Technology: The Liberated Learning Consortium. In S Dawe (Ed.), Vocational Education and Training and Innovation: Research Readings, (pp. 212-225). Adelaide, South Australia: National Centre for Vocational Educational Research.

Paez, D., & Raciti, M. (2002). Bridging the gap: Speech recognition technology supporting first year students during transition to higher education. Proceedings of 6th Pacific Rim, First Year in Higher Education Conference 2002: Changing Agendas-Te Ao Hurihuri, Christchurch, New Zealand.

Thompson E.S. (2003). Text-structuring metadiscourse, intonation and the signaling of organization in academic lectures. Journal of English for Academic Purposes, 2, 5-20.