Power at my fingertips but which button do I press? Researching and evaluating learner interactions in hypermedia

The development of optical technologies with increasing storage capacities has led to products using hypermedia. As these products are distributed to industry and education there is necessity for evaluation of them to ascertain if they really do enhance the learning experience by providing the learner/user with flexible tools, simple interfaces and appropriate control.

This paper identifies the research issues of homogeneity; position in the information space; appeal to the user; pathways of learning; influence of prior problem solving approaches and learning and features that need to be provided for the user and level of interactivity relative to the product's instructional aims.

An enhanced case study approach (using video as a data collection instrument) is proposed as one method of evaluating hypermedia and learner interactions. This paper discusses experiences and results of a pilot study utilising this approach.

Introduction

An indication of the proposed impact of CD-ROM technology is shown in the CD-ROM drives sales forecasts. Worldwide sales figures released by Sony show an increase from less than one hundred thousand units in 1988 to over one million units in 1991 (Fox 1990) and commercially available CD-ROM titles of less than two hundred in 1988 to over two thousand five hundred in 1991 (Nicholls 1990).

Following CD-ROM's establishment in the commercial world of computing, a number of products have become available for the educational market and an increasing number of tertiary institutions are developing CD-ROM based applications for the enhancement and development of courses. These packages offer the user a variety of features and interfaces.

CD-ROM and hypermedia

Hypermedia: Some issues

These tools give access to search techniques and auxiliary features such as graphics, sound, tutorial, screen design and links. They allow sorting, rearranging, merging information and connecting together facts into a knowledge structure - one that is linked by associations. These create an environment which has the potential to enhance the quality of learning, making it fear-free. There is also the opportunity to practice skills; utilise search paths as determined by the user (Ofiesh 1990); allow alternatives to traditional paths of learning, and use techniques more compatible with independent learning styles.

Facilities do not necessarily equate with learning. Copeland (1991) in the paper 'The Multimedia Mix" suggests that with multimedia there is an opportunity for increased interaction between the user and the resource but there could be a threshold of interactive activity after which increasing the level further could be counter productive. Copeland suggests that research is required in this area to increase our understanding of the nature of interactive activities.

Hedberg (1989) says that CD-ROM poses challenges for new and alternate learning/teaching strategies and that the learner must know how information can be retrieved and strategies for searching digitised information developed. This view also suggests the use of research utilising qualitative and enhanced case study methodology.

Finally, Hall (1991) in developing a hypermedia program using hypercard and videodisc for a biology program received the following responses from participants who used the material. Participants overwhelmingly enjoyed the learning experience, although some became disorientated and indicated difficulties in navigating around the information. Ibis finding further supports the need for investigation of learner interaction and navigation strategies through hypermedia.

Hypertext: A powerful tool

Issues in using hypertext

Some designers of hypertext material have addressed these problems with the argument that a book metaphor is the best approach. They maintain that it gives the reader/user a familiar format. An argument against this perspective however is that such a metaphor seems to limit the conceptual models of the search potential of hypertext and non-linear navigation of the information space (Nielsen, 1990).

In education we must go beyond understanding CD-ROM as a large storage platform and hypermedia as a combination of digitised information, and seek to identify the features which will increase our understanding of design factors important in the development of these materials and thus enhance the teaching/learning interaction.

Many questions arise in the educational context of hypermedia and CD-ROM.

• Do multimedia, CD-ROM and their associated enhance learning?

• What tools best provide for the navigational needs of users?

• What is the impact of multimedia and hypertext upon the user in the teaching/learning interaction?

• Is the learner being given adequate control of their learning and being provided with the appropriate tools?

• What tools are supportive in providing the learner with the control and access that is required to realise the power of the materials?

• What can we learn by a closer examination of the user interacting with these products?

• Will enhanced case study methodology provide a way of evaluating the teaching/learning interaction?

The enhanced case study approach

The case study

Program description

In these materials the user is presented with a graphical interface consisting of twelve tools which interact with the material. The icon bars twelve functions are: image (go to an image and return from it), index, notebook (word processor), print, search, next screen, previous screen, return to link anchor, screen history, contents, help and exit system. Icons similar to those used in the materials are shown in fig 1.

Figure 1: The icon bar's functions

Subject

Figure 2: Workstation setup

Method

The video equipment was superimposed on the workstation as a second component of the research environment (figure 3). Two video images were recorded simultaneously, one image full screen and the second windowed in that screen to 1/4 size. The full size screen recorded the detail of the computer screen. This approach gave a history of the pathway the student selected for moving through the data by clearly identifying the screen location number in the bottom left corner. Accurate placement of the cameras was required to capture the screen fully and gain a clear picture free of distortion and to avoid any obstructing by body movement of the subject.

Figure 3: Workstation setup for video taping

One of the cameras was configured especially for recording computer screens and removed any 'flicker'.

2. The interview

On completion of the sessions the subject was interviewed. This was recorded on video with the interviewer off camera asking prepared questions. These questions sought clarification of responses observed during the sessions and sought the subjects opinion on features and appeal of the materials.

3. Data coding and structure

Each screen within Linkway identifies the folder and the screen number which is allocated at the time of construction. This information was positioned at the bottom left hand corner of the screen and used the format [Program Name: Folder Number: Screen Number]. This was coded into a three digit code; the first number being the folder and the subsequent digits identifying the screen within that folder. This data was transcribed from the video record and entered into a word processor. To enable charting of the data, it was copied into a spreadsheet and the 'graph' option executed.

Sessions

Session 1

The video indicated that little exploration of other icons occurred until later in the session with the image functions being used. The more powerful functions of searching and following links within the material were ignored.

The graph (figure 4) demonstrates the progression of the subject during this session with time on the x axis and screen identification on the y axis. As has been described in the Data Analysis section the screens were coded for analysis with a three digit number, the first being the relevant folder in Linkway (2) and the next two digits identifying the screen within that folder.

Figure 4: Session 1 data

Session 2

The graph (figure 5) plotting screen identification against time details the linear investigation of material. Towards the end of the session there is greater movement through the various folders of the program displaying more experimentation and some use of hypertext links by the user.

Figure 5: Session 2 data

Session 3

Figure 6 demonstrates the amount of movement which occurred during the session. In comparison to previous sessions a great deal of vertical movement through the application can be seen. This movement indicates choice of a topic, proceeding to a screen and the selection of associated links.

Figure 6: Session 3 data

The subject also made use in this session of the index icon and search function to identify the topic, explore the links within the topic (either picture or text) and record comments in the notebook (a text editor).

A recurring difficulty was that the subject tried to cover all topics and used the index to check progress. As there was no log of navigation of her recording where she had been, she commonly retraced steps unnecessarily. The need for a navigational log which clearly identified paths taken and nodes explored was apparent and that the navigational functions proved inadequate.

Insight into subject's learning

Background

The response by the subject when she reached familiar content in the program and the reinforcement and encouragement that occurred at this point, highlighted the link between the current learning episode and her past experiences and understandings. An example of this was when she discovered that the 'notebook' icon took her into a simple word processor with which she had some previous experience.

Intuition was used as the subject concluded that "..there must be an easier way to do this!" By this stage she seemed to intuitively know that the program material would offer a more direct or powerful way to get to the information she wanted to reach. It could be suggested by this incident that a Hawthorne effect might have come into play at this stage. This might also indicate the development of an enhanced problem solving strategy. Intuitive learning now became part of her learning strategy.

As familiarity increased with the features of the program the need for more powerful navigation features increased. It was at this stage that the inclusion of a map listing previous paths would have prevented the subject from going where she had already been.

Finally, the progression described in these sessions demonstrates that there was an amount of time required for the subject to develop strategies appropriate to the use of these material and that the tools or functions provided were not enough in themselves. The process of learning from hypermedia in this case required development of strategies and understandings by the subject before the features of the program could be utilised. On completion of the sessions the subject expressed enjoyment in using the application. A desire was expressed for more of this type of material.

Approaches to learning from hypermedia

A feature of the approach the subject displayed was the ability to continue on if something was not understood. Her verbal comments indicated that she was confident with the thought she would return to it at a later stage with added insight and be able to understand it.

Of further interest was the experimentation which allowed the subject's to investigate a number of features of the interface. This feature of her approach was characterised by the question "What happens if I do....?"

More generally, the subject's movement was from a very linear 'what comes next' type approach to a much more encompassing or diverse approach. This movement to this new strategies was a process which developed as she became more confident in altering and developing the navigational structures of the materials. She was gradually able to adapt the attitudes and skills that she had bought to the sessions.

Summary and conclusions

The design and development of hypermedia materials may be enhanced by:

• the inclusion of known material/functions/tools (such as the word processor) to assist the user in moving into unfamiliar concepts or material.

• the design of features such as location maps which allow users/learners to locate their position at any point in time and retrace their steps.

• the provision of means of connectivity between information and catering for individual learning styles and paths. Users should be able to make links within the information and delete links and nodes that are not functional for them - to change the database structure.

• responsiveness to varying user levels ie. the provision of appropriate tools and views for novice and expert users, and which users can modify to suit their requirements.

• the avoidance of 'cluttering' the application with complex features which eventually have a negative impact on the teaching/learning interaction.

• Interview structures must be precise.

• Questions must be pivotal and focused.

• Data structures should be organised and coding configurations established well in advance of data collection. This can be done in conjunction with product design in the establishment of a dump file or record file.

• Transcription from videotape is both time consuming and demanding in the maintenance of accuracy. The use of precision time coding imprinting may enhance the accuracy and ease of collation and analysis.

• A three camera configuration is preferable to two cameras in determining the finer points of subject interaction and the transparency of the interface design.

• A personal microphone rather than a directional microphone produces more consistent audio as subjects ' think aloud responses change in volume. This should be considered in association with considerations of 'how much' recording equipment should impose in the research environment.

• Split screen is preferably used with a wide angle view which allows vision of the entire working area. (Occasionally in this study the 'mouse pointer` ranged out of view of the camera.)

• A camera which eliminates screen flicker from the computer screen allows a sharper, more precise image which facilitates more accurate analysis.

• While the media lends itself to long and intense periods of interaction, session time limits of about 30 minutes proved far more productive than those in which no time limits were imposed.

• Provision of practice or technology familiarisation sessions overcomes initial difficulties with the 'mouse' or interface.

• Collect the jottings of the subject - they may contain useful additional information.

Where to from here?

Notes

1. Node is... [ sentence incomplete in the original ]
2. Linkway is hypermedia authoring software for IBM and compatibles

References

Fox, A. E. (1990). CD-ROM hardware: What's real, what's next?, Online/CD-ROM'90 Conference Proceedings, 63-67.

Hall, W., Thorogood, P., Sprunt, B., Carr, L. and Hutchings, G. (1990). Is hypermedia an effective tool for education? Computers in Education. North Holland: Elsevier Science Publications.

Hedberg, J. G. (1989). CD-ROM: Expanding and shrinking resource-based learning. Australian Journal of Educational Technology, 5(1), 56-75. http://www.ascilite.org.au/ajet/ajet5/hedberg1.html

Kidder, L. & Judd, C. (1986). Research Methods in Social Relations. NY: Holt, Rinehart and Winston.

Megarry, J. (1988). Hypertext and compact discs: The challenge of multimedia learning, British Journal Educational Technology, 19(3), 172-183.

Nicholls, P. T. (1990). CD-ROM Collection Builder's Toolkit. Pemberton, Weston, CT, 1990, 3.

Nielsen, J. (1990). Through hypertext. Communications of the ACM, 33(3, March), 297-310

Ofiesh G. and Murphy, B. L. (1990). As we see it: CD-ROM and education. CD-ROM Enduser, 2(5, September), 26-29.

Oppenheimer, C. (1988). CD-ROM: Fundamentals to Applications. London, Butterworths.

Park, O. (1991). Hypermedia: Functional Features and Research Issues. Educational Technology, August, 24-31.

Please cite as: Webster, L. L. and McNamara, S. E. (1992). Power at my fingertips but which button do I press? Researching and evaluating learner interactions in hypermedia. In Promaco Conventions (Ed.), Proceedings of the International Interactive Multimedia Symposium, 285-297. Perth, Western Australia, 27-31 January. Promaco Conventions. http://www.aset.org.au/confs/iims/1992/webster.html