ASCILITE 2004: Bennett, Lockyer and Agostinho - learning designs as a framework to incorporate learning objects

[ ASCILITE ] [ 2004 Proceedings Contents ]

Investigating how learning designs can be used as a framework to incorporate learning objects

Sue Bennett, Lori Lockyer and Shirley Agostinho
Research Centre for Interactive Learning Environments (RILE)
University of Wollongong, Australia

Much of the current research and development efforts related to learning objects focus on the concept of reuse from a technical perspective, particularly in terms of technical standards for interoperability. Yet, there is little research being conducted that investigates how learning objects can be reused from a pedagogical perspective. It is necessary to develop a deeper understanding of the processes teachers undertake to make use of learning objects in their teaching. This paper reports the current progress of a project that is addressing this gap by investigating how tertiary teachers make use of generic learning designs as a framework for incorporating learning objects into their subjects.

Introduction

The trends toward the commercialisation of education and an interest in collegial sharing of ideas and content have shifted the e-learning design and development agenda to focus on making the most of existing and new resources by reusing them in multiple settings. This shift has given rise to the concept of 'learning objects'. These are digital educational resources that can be stored in repositories and described in such a way that they can be easily identified and reused by educators in different learning contexts (Downes, 2001; Wiley, 2002).

In a general sense, this idea is not new. Historically teachers have reused educational ideas and resources previously developed by themselves or others. However, the learning object concept extends the scale of this activity by making resources available to teachers worldwide via the Internet. To facilitate this process, learning objects must exist in an appropriate form, they must reside in locations where they can be easily accessed, and must be accompanied with an annotation to facilitate their retrieval.

Much of the work on learning objects to date has focused on technical aspects of the storage and retrieval process (Bannan-Ritland, Dabbagh, & Murphy, 2002; Hand, Gosper, Woo, Gibbs, Kerr, & Rich, 2004). Limited attention has been paid to the pedagogical and practical factors affecting reuse. Nichols (2002) claims that the emphasis of learning object research and development has been on the 'object' and not on the 'learning.' Wiley (2002) has warned that if this shortcoming is not addressed soon "we will find ourselves with digital libraries full of easy to find learning objects we don't know how to use" (p.1).

Recent literature and forums about the use of learning objects indicate that, whilst there has been much activity in establishing technical infrastructures to facilitate the use of learning objects, their actual uptake by practitioners is still in its infancy (Johnson, 2003; Griffith, 2003; Hand et al., 2004). In 2002, a forum of American, Canadian and Australian researchers and practitioners concluded that the current level of learning object activity could not be described as "pervasive" (Johnson. 2003, p.7). This is supported by a survey conducted in the United States higher education sector that found the use of learning objects to be in an introductory stage rather than a widespread practice (Griffith, 2003). More recently, a panel discussion at the 2004 EdMedia conference concluded that there were still significant barriers to teachers' use of learning objects.

A factor identified by Griffith (2003) for the lack of uptake is a lack of pedagogical models that make learning objects effective: "Models for 'best practices' are constantly evolving, but that process would be greatly enhanced by increased applied research and evaluation in this area" (p.14). Thus, there is a need for a better understanding of the process by which teachers construct learning experiences and consider using resources to support those experiences. Such understanding will provide a basis for developing strategies and frameworks to support teachers to make effective use of electronic resources, including learning objects.

The use of generic forms of learning activities, or 'learning designs', has been proposed as an approach to support teachers and designers in developing learning experiences that incorporate learning objects (Agostinho, Bennett, Lockyer & Harper, 2003). This idea is in keeping with Wiley's (2003) suggestion that learning objects be considered as resources used within in a broader learning experience, and the argument that learning objects should support learning activities and it is the activity that should drive the selection and inclusion of learning objects (Jonassen & Churchill, 2004). This perspective is also supported by a recent study that explored how practitioners from three educational sectors in Australia currently use learning objects and found that teachers view learning objects as "just another resource that can be called upon to contribute to the development of curricula and to assist in the process of teaching and learning" (Hand et al., 2004, p.71).

Some work has begun to identify and document effective learning designs (Agostinho, Oliver, Harper, Hedberg & Wills, 2002; Laurillard & McAndrew, 2003). A learning design describes a sequence of activities, resources and interactions that comprise a particular learning experience, for example a lesson, module or subject. A generic learning design describes the sequence and nature of activities and accompanying resources and supports in a context , discipline and content independent manner. It is proposed that a generic learning design could serve as a pedagogical framework to support teachers in creating learning experiences, with the teacher adapting the learning design, specifying the particular activities, and choosing or creating the resources and supports needed to suit his/her learners.

This approach seems consistent with the concept of learning objects as discrete units of learning material that can be reused within different learning contexts. As yet, there has been little research conducted to investigate the feasibility of such an approach. Issues such as whether teachers are able to interpret generic learning designs, how they identify particular learning designs as appropriate for their context, how they adapt and apply generic learning designs, and how they integrate learning objects into learning designs requires exploration.

This paper reports on a project that seeks to address this gap in understanding by investigating how tertiary teachers make use of generic learning designs as a framework for redesigning their subjects and incorporating learning objects. The study aims to:

Identify the pedagogical and practical factors that support and constrain use of generic learning designs;
Characterise the processes and decision making steps involved in selecting and integrating learning objects; and
Determine the role that learning designs can play in supporting effective use of learning objects.

This paper reports on the findings of the project to date, which are associated with the first aim. Further research currently in progress will address the further aims of this study.

Method

Research approach

As learning objects are not yet widely used in the tertiary sector, the study could not be conducted as an observation of existing practice. Instead, data was collected in a workshop setting in which the participants were introduced to the concept of learning designs and then supported as they applied these to their own teaching contexts. This approach has enabled the researchers to explore the changes in the participants' ideas about incorporating electronic resources and to investigate the process by which the participants designed, developed and implemented learning experiences based on the learning design and learning objects they chose. The process was facilitated by the research team, but was open ended to allow participants to explore issues relevant to their teaching contexts.

Context

The participants in the study included four tertiary teachers who aimed to re-design an undergraduate pre-service teacher education subject that services four discipline specialisations (Primary, Early Childhood, Physical and Health Education, and Secondary Maths/Science Education). This subject was considered well suited for this study because, although the overall structure of the subject would be consistent across all four disciplines, the design could be customised to suit the different needs of each group.

The subject focuses on teachers' use of information and communication technologies for both personal and professional purposes. The subject is offered over a 13 week teaching session and is compulsory for all students undertaking the Bachelor of Teaching degree. Previous evaluations and a review by the staff involved in teaching the subject identified limitations related to professional application. Its existing design introduced students to a range of technology skills however these skills were not necessarily applied to the professional context in which the learners would eventually find themselves (i.e. school classrooms). Thus, the study participants were seeking a more appropriate learning design for the subject.

The complexities of the subject are that:

it caters for students across four different discipline specialisations (early childhood education; primary education; secondary physical and health education; and secondary maths and science education);
the specialisation cohorts vary in size from 20 to 200; and,
the subject is offered in either the first or second year of the program depending on the specialisation.

The study participants wanted to retain the overall subject objectives and agreed it was necessary to have consistency in the assessment tasks across the subject, but also wanted to enable the students to apply the knowledge and skills they learned to the context of their specialisation. All participants expressed an interest in investigating problem based learning as a potential teaching and learning strategy, though none had previous experience in designing and implementing such an approach.

Design and development activities

The generic learning design templates utilised in this study come from the Information and Communication Technologies and their Role in Flexible Learning project, an Australian government funded project that identified a collection of learning design exemplars demonstrating innovative use of information and communication technologies. A selection of these designs was further abstracted into generic forms as scaffolds for designers and teachers. Textual descriptions and visual representations of the learning designs provide information about the learning activities, resources and supports. Details of the project and descriptions of the exemplars and generic learning designs are published at http://www.learningdesigns.uow.edu.au/.

Three learning designs were identified as possibilities to help the participants think about how the subject could be re-designed with a problem based approach. The three generic learning designs selected were:

Explore, Describe, Apply: A problem focussed learning design
Review, Access, Question, Decide, Report, Reflect: Structured problem solving
Review, Interpret, Construct, Justify: A situated problem focussed learning design

For each learning design, documentation for the generic model and a contextualised exemplar was provided to the participants in hard copy prior to the workshop. Participants were asked to review the learning designs before attending. The participants were also provided with a list of the resources (learning objects) which were available from previous offerings of the subject.

During the workshop, the participants:

Discussed their assumptions about the subject and the needs of their specialisation, brainstormed its limitations and strengths, and stated what they wanted to achieve from the re-design;
Explored a variety of generic learning designs based on problem solving approaches in a one on one break out session with members of the research team;
Reviewed a list of available resources (learning objects);
Developed an overview of their preferred approach;
Shared their design ideas with the other participants; and
Devised an overall approach for the subject.

After the workshop versions of the subject were developed for the Early Childhood and Physical and Health Education specialisations for implementation in Spring session (July-November 2004). These implementations will be evaluated to inform the development of the subject for the Primary and Secondary Maths/Science specialisations to be offered in Autumn session 2005 (March-June 2005).

Data collection

Data was collected before, during and after the workshop in the variety of forms, as shown in Table 1. The final column indicates the status of data collection in July 2004.

Table 1: Data collection

Stage Data collection Rationale Research
sub-questions Status

Prior to the workshop Participants were interviewed to collect background information about them and their experiences with electronic resources. This will help the researchers to characterise participants' teaching/design practice, technical skills and pre-conceptions about electronic resources. What are participants' initial ideas about using electronic resources in teaching?
What prior experiences do they have in using or designing with electronic resources? Completed

During the workshop Data collection was integrated into the workshop program. Data was collected in the form of overt observations, records of discussions, and artefacts produced by the participants. Data collected will be used to investigate the process by which participants interpreted the learning designs provided, selected a design to use and their initial ideas for implementing that design. What criteria do participants use when deciding to use a learning design?
How do participants adapt a learning design to suit their particular context? Completed

After the workshop Through semi-structured interviews, participants will be asked about their ideas about learning designs and objects, about the process by which they developed their designs and selected the resources, and how they implemented their versions of the subject. Planning documents, team meeting notes and the final set of subject files for each subject offering will also be collected for analysis. Data from this stage will provide evidence about: how the learning design approach has influenced the participants' ideas about redesign, how they used learning objects, in particular the factors that support and constrain use/re-use; and whether the learning design framework assisted the design process. How did a learning design approach assist participants?
How do participants identify required learning objects?
How did participants' ideas about using electronic resources in their teaching change?
How does a generic learning design framework support teachers in making use of learning objects? To be conducted from July to November 2004.

Preliminary findings

Data from the initial interviews and workshop session has been collected and a preliminary analysis has been undertaken. Further analysis will be undertaken after November 2004 when data collection is complete.

Participant profiles

All four participants had considerable experience in teaching within a university context (ranging from 10 years part time experience to 16 years of full time experience). Three of the four participants also had extensive experience as school teachers (ranging from 15-28 years of service).

All participants were accustomed to using the Web in their teaching - particularly within the context of a learning management system. They had all, at some time, created a web based environment themselves. One participant did so with support from instructional designers. All were familiar with communication technologies and used email to communicate with their students - some also used asynchronous discussion tools within the learning management system.

All participants used a range of electronic resources in their teaching. These tended to be in the form of subject outlines, lecture notes, and readings and links/references to other web sites. Some used past student work as examples and online quizzes.

Curriculum design practice

All participants engaged in some form of re-design of the subjects they teach on an annual basis. They reported that they tended to base their re-design decisions on student feedback and analysis of and alignment with school based syllabus developments. Three of the four participants explicitly stated that when designing or re-designing their subjects their foremost consideration is meeting the needs of the students.

In terms of deciding on an overall structure for a particular subject, the participants' responses differed somewhat. One participant stated that the main challenge was to determine a model or framework that represents sound pedagogy and which they felt comfortable to implement. Another participant said that the time available for the re-design was a factor in design decisions. One participant responded that design decisions were based on professional judgement about the most logical way to present the subject. The fourth participant gave a more content focused response stating that the structure of a subject depended on the knowledge students need to learn.

Resource use and reuse

Three of the participants indicated that they would be prepared to use resources created by others. One participant has been mainly involved in subjects where resources were created by others but liked to supplement these resources with ones that the participant had created or found. Another participant, stated what whilst they would be happy to use resources created by others, they had mostly used self developed or self identified resources. The electronic resources that were routinely created by the participants included lecture notes and general content supportive materials such as demonstrations (e.g., spreadsheets, slide presentations and multimedia demonstrations) of how technology can be used by teachers. The participants stated that they usually find electronic resources by searching the Web, asking colleagues and accessing library databases.

Negotiating an overall design

Overall, the Explore, Describe, Apply (EDA) learning design was deemed most suitable for the subject as an overarching framework. Two participants chose EDA as the most appropriate of the three examples. The other two, whilst not immediately identifying the EDA, felt it could be applied for the re-design of the subject.

However, all four participants had their own ideas about how the subject should be re-designed. Thus, the particular learning activities to be conducted in each phase of the learning design differed considerably for the four specialisations and also from the description of the generic learning design. As such, while the three phases of Explore, Describe, Apply were maintained, the participants wished to adapt the particular activities in each phase of the learning design.

The participant's knowledge of the student profile informed their design decisions. The participants agreed that students tend to have little experience of the classroom context given they are at the early stages of their program. Also, while some students are proficient with computers, many have limited technical knowledge and skills. Thus, there was consensus among the participants that some direct teaching in the first phase of the subject would be beneficial. This led to an overall design in which lectures and tutorials would be conducted during the first six weeks of session to introduce concepts. This would be followed by a seven week period during which students would work on a problem based project based on the EDA learning design. Thus, the participants chose to scale down the EDA design to embed it within a subject, rather than using it for the design of the whole subject.

Interpreting the generic learning designs

Discussion amongst the participants within the workshop identified that generic learning designs were difficult to interpret as a stand alone resource. The contextualised exemplar from which the generic learning design was abstracted was needed as accompanying documentation to help the participants make sense of the learning design. The participants found that the generic learning design documentation served more as an overall model. To understand the detail of what students would actually do and how content resources are used to support them participants needed to consult the contextualised learning design for an example of application.

What also surfaced is that, whilst the generic learning designs are expressed broadly in terms of content or discipline focus, they are quite specific in terms of process (i.e., the types of activities students are required to do). This was another reason why participants found it difficult to directly translate the generic design to the content of their subject. Thus, the generic learning designs served as ideas for providing an overarching structure of the subject even though the particular activities were not appropriate and had to be adapted.

It was also observed that the visual representation of both the generic and contextualised learning designs were used as the focal point of discussion in both the one on one conversations and within the whole group negotiations. This indicates that once the design was understood the visual format provided a useful overall model.

Discussion

Although this research is still in its early stages and at present is limited to a small number of participants, it appears that learning designs are useful as a trigger for thinking about how new strategies might be applied to an existing subject design. The findings also confirm the usefulness of specific design examples that help designers put those ideas into context. Additional examples of the application of a learning design may further enhance this process. It may also be that more investigation is needed into how generic versions of learning designs are abstracted from the contextualised exemplars such that the activities can be described in a more general way. Exploration of how visual representation of the activities, resources and supports embedded in both generic learning designs and contextualised exemplars serve as a scaffold to the design process may further this approach.

The scope of the study is limited in that it investigates only generic learning designs related to problem based learning, the application of generic learning designs to education subjects and this use of generic designs by experienced educators. Differences in the genre of the generic learning design used, the discipline in which it is applied and the experience of the designer or teacher may impact differently on the process. The researchers plan to explore some of these issues in further work. In addition, this research is based on the generic learning designs derived from the Information and Communication Technologies and their Role in Flexible Learning project. As such the findings may not be applicable to other frameworks that use different formats, such as generic learning activities as discussed by Laurillard & McAndrew (2003). This also requires further investigation.

The next stage of the research will involve the researchers in analysis of the data collected during the development and implementation phases of process. This stage will provide some insights into the use of learning objects by the participants and further explore their ideas about this approach to design.

Conclusion

This study has begun to make a significant contribution to the knowledge base in an emerging area of educational technology. Preliminary findings indicate that generic and contextualised learning designs are useful in the initial phase of a subject redesign and further analysis will reveal more about how they support the integration of learning objects. By investigating teachers'/designers' perspectives, this study begins to address the imbalance between research on the technical aspects of learning object use and research on the pedagogical and practical aspects. Findings from the study will provide a basis for further research, as well as informing practice and technical development.

Acknowledgement

The work reported in this paper comprises a research study funded by the NextEd ASCILITE Research Grant awarded at ASCILITE 2003. The authors wish to thank NextEd and ASCILITE for making this research possible.

References

Agostinho, S., Bennett, S., Lockyer, L., & Harper, B. (2003). Integrating learning objects with learning designs. Interact, Integrate, Impact: Proceedings 20th ASCILITE Conference, pp. 571-575. Adelaide, 7-10 December. http://www.ascilite.org.au/conferences/adelaide03/docs/pdf/571.pdf

Agostinho, S, Oliver, R., Harper, B., Hedberg, J., & Wills, S. (2002). A tool to evaluate the potential for an ICT-based learning design to foster "high-quality learning". In A. Williamson, C. Gunn, A. Young & T. Clear (Eds.), Winds of change in the sea of learning. Proceedings 19th ASCILITE Conference, pp. 29-38. Auckland: UNITEC Institute of Technology. http://www.ascilite.org.au/conferences/auckland02/proceedings/papers/164.pdf

Bannan-Ritland, B., Dabbagh, N., & Murphy, K. (2002). Learning object systems as constructivist learning environments: Related assumptions, theories, and applications. In D. A. Wiley (Ed.), The Instructional Use of Learning Objects (pp. 61-98). Bloomington, Indiana: AIT/AECT. [verified 9 Oct 2004] http://www.reusability.org/read/chapters/bannan-ritland.doc

Downes, S. (2001). Learning objects: Resources for distance education worldwide. International Review of Research in Open and Distance Learning, 2(1). [viewed 11 April 2003, verified 9 oct 2004] http://www.irrodl.org/content/v2.1/downes.html

Griffith, R. and Academic ADL Co-Lab Staff (2003). Learning Objects in Higher Education. Academic ADL Co-Lab with Sponsorship by WebCT. [verified 9 Oct 2004] http://www.academiccolab.org/resources/webct_learningobjects.pdf

Hand, T., Gosper, M., Woo, K., Gibbs, D., Kerr, S., & Rich, D. (2004). Learning objects: User perspectives on the conditions surrounding their use. Proceedings of the ED-MEDIA 2004: World Conference on Educational Multimedia, Hypermedia and Telecommunications, 21-26 June, Lugano, Switzerland, pp. 66-72. AACE.

Jonassen, D., & Churchill, D. (2004). Is there a learning orientation in learning objects? International Journal on E-Learning, 3(2), 32-41.

Johnson, L. F. (2003). Elusive Vision: Challenges impeding the learning object economy. Macromedia. [verified 9 Oct 2004] http://download.macromedia.com/pub/solutions/downloads/elearning/elusive_vision.pdf

Laurillard, D. & McAndrew, P. (2003). Reusable educational software: A basis for generic learning activities. In A. Littlejohn (Ed), Reusing Online Resources: A Sustainable Approach to E-learning (pp. 81-93). London: Kogan-Page.

Nichols, M. (2002). Education and Learning Objects: A Primer. [viewed 13 Mar 2003, not found 9 Oct 2004] http://www.cta.ac.nz/papers/Learning%20Objects%20CPIT.doc

Wiley, D. A. (2002). Learning objects. In A. Kovolchick & K. Dawson (Eds), Educational Technology: An Encyclopaedia. Santa Barbara: ABC-CLIO. [viewed 7 Apr 2003, verified 9 Oct 2004] http://wiley.ed.usu.edu/docs/encyc.pdf

Wiley, D. A. (2003). Learning objects: Difficulties and opportunities. Academic ADL Co-Lab News Report: No. 152-030406. [viewed 7 Apr 2003, verified 9 Oct 2004] http://wiley.ed.usu.edu/docs/lo_do.pdf

Please cite as: Bennett, S., Lockyer, L. & Agostinho, S. (2004). Investigating how learning designs can be used as a framework to incorporate learning objects. In R. Atkinson, C. McBeath, D. Jonas-Dwyer & R. Phillips (Eds), Beyond the comfort zone: Proceedings of the 21st ASCILITE Conference (pp. 116-122). Perth, 5-8 December. http://www.ascilite.org.au/conferences/perth04/procs/bennett.html

© 2004 Sue Bennett, Lori Lockyer and Shirley Agostinho
The authors assign to ASCILITE and educational non-profit institutions a non-exclusive licence to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. The authors also grant a non-exclusive licence to ASCILITE to publish this document on the ASCILITE web site (including any mirror or archival sites that may be developed) and in printed form within the ASCILITE 2004 Conference Proceedings. Any other usage is prohibited without the express permission of the authors.

[ ASCILITE ] [ 2004 Proceedings Contents ]
This URL: http://www.ascilite.org.au/conferences/perth04/procs/bennett.html
HTML created 19 Nov 2004. Last correction: 10 Apr 2005.

Stage	Data collection	Rationale	Research sub-questions	Status
Prior to the workshop	Participants were interviewed to collect background information about them and their experiences with electronic resources.	This will help the researchers to characterise participants' teaching/design practice, technical skills and pre-conceptions about electronic resources.	What are participants' initial ideas about using electronic resources in teaching? What prior experiences do they have in using or designing with electronic resources?	Completed
During the workshop	Data collection was integrated into the workshop program. Data was collected in the form of overt observations, records of discussions, and artefacts produced by the participants.	Data collected will be used to investigate the process by which participants interpreted the learning designs provided, selected a design to use and their initial ideas for implementing that design.	What criteria do participants use when deciding to use a learning design? How do participants adapt a learning design to suit their particular context?	Completed
After the workshop	Through semi-structured interviews, participants will be asked about their ideas about learning designs and objects, about the process by which they developed their designs and selected the resources, and how they implemented their versions of the subject. Planning documents, team meeting notes and the final set of subject files for each subject offering will also be collected for analysis.	Data from this stage will provide evidence about: how the learning design approach has influenced the participants' ideas about redesign, how they used learning objects, in particular the factors that support and constrain use/re-use; and whether the learning design framework assisted the design process.	How did a learning design approach assist participants? How do participants identify required learning objects? How did participants' ideas about using electronic resources in their teaching change? How does a generic learning design framework support teachers in making use of learning objects?	To be conducted from July to November 2004.