Full Paper

Tom Fenton-Kerr, Tony Koppi, Marcel Chaloupka, Stephen Cattle, Alison Anderson

tfk@nettl.usyd.edu.au

NeTTL, Centre for Teaching and Learning, University of Sydney

Interface design in its most general sense covers issues as broad as: presentation of mixed media forms, knowledge navigation, and the provision of support in the form of on-line help files, guides and software agents. These support items constitute a range of software programs that in an educational context are designed to aid the completion of a learning task in some way. Collectively, they form what we term an ALLIANCE (an Agent for Leveraged Learning In A Networked Collaborative Environment) which is able to undertake a range of tasks, each of which adds to the possibility of success in a given field.

We should first distinguish what we mean by a software learning agent in this particular context. Here, the phrase is employed in the sense of a piece of software used for an educational purpose, as opposed to software specifically designed to learn or acquire new knowledge by itself. This is not to suggest that certain manifestations of agent-based software are incapable of such acquisition: in fact, many learning agents must adapt and 'learn' if they are to be capable of serving their users' learning needs effectively.

The Macquarie Dictionary defines an agent is "a person acting on behalf of another", or "a natural force or object producing or used for obtaining specific results". While software agents could hardly be classified as "natural", they are certainly used for obtaining results, and, in the case of autonomous learning agents acting remotely, can be said to be acting on behalf of another entity, such as a learner, teacher or other software. IBM's White Paper, The Role of Intelligent Agents in the Information Infrastructure (Grosof et al., 1995) uses an interpretation that is close to the dictionary definition of agency mentioned above, in a rather more specialised context, defining agents as:

... Software entities that carry out some set of operations on behalf of a user or another program with some degree of independence or autonomy ...employ[ing] some knowledge or representation of the user's goals or desires.

This latter definition fits within our concept of a software learning agent, incorporating, as it does, the elements of agency, intelligence, and in certain circumstances, mobility.

Wooldridge and Jennings (1995:2) describe software agents as having traits such as autonomy, social ability, reactivity, and pro-activeness. Reactivity is a key defining trait for Wooldridge (1997:48), i.e., the ability to "modify its behaviour as appropriately as environmental circumstances change", along with some sense of goal-directed behaviour is what distinguishes an agent from other software programs that may use structured programming or are designed to be user friendly.

Taxonomically, software learning agents fall within the domain of artificial intelligence (AI) as applied to educational settings. An overall view of software agents, classified according to their functionality, might include the following:

• collaborative agent systems that mediate and facilitate interactions between users
• information retrieval agents that can be configured for specific searches, and are
• usually autonomous agents that are able to take action independent of input
• capable of autonomous action
• intelligent tutors or expert systems based on an expert (human) model, capable of adaptive behaviour and able to a user to some suitable outcome

Obviously, there is a large amount of overlap between the functional classification of agents mentioned above. A learning agent that acts autonomously in searching a network on a user's behalf and then presents its findings in an adaptive format makes use of several of them. In doing so, it may need to call on other agents, resources and algorithms as represented in Figure 1., below. One element that appears to be common to all learning agents, however, is their position on the learning map: between the user and the information required to complete a learning task.

To summarise; a software learning agent is a piece of software that acts in an intelligent fashion to assist a user with a learning task. In so doing it may display autonomous behaviour, react in a useful way to user input, and sometimes act on a user's behalf (for example, at a remote location). Some agents may be capable of pro-active behaviour in the attainment of an educational goal. Agents may also display human-like traits such as desires, motivations, beliefs about a world-state, and personality characteristics. These latter characteristics may be represented to the user in graphic, aural or text forms.

One of the basic questions this research is attempting to answer is: can software programs such as learning agents and their variants be created that assist learning in a problem-based task? The same question on an analytic level might be: what would such software have to do to accomplish such a task? Suggestions include:

• meeting and accounting for learning goals: Learning objectives should be clearly understood and stated before implementing software assistance. Evaluation of a learning assistance program should, at the very least, query whether learning objectives have been met, and how much influence the program had on the outcome.
• Assisting with, and responding to, student strategies: A learning agent should have enough information about a given user's current level of understanding and position on the learning map to offer contextual advice about educational content or navigational aspects of the program. Recent work done by Conati et al. (1997) focuses on coached problem solving where the user-tutor interaction may change as the user progresses through a program. Their intelligent tutoring system (ANDES) teaches Newtonian physics to naval undergraduates. The tutor's interaction with the user depends on the user's level of understanding at any given point in the program. Satisfactory progress by a student elicits little response from the tutor other than encouragement. When a student strays off the track or gets stranded at some level, the tutor can step in to offer tailored hints that indicate the best solution path to follow.
• developing a capacity for learning (via metacognitive revelation) in students: As with any good scaffolding approach, a learning agent should support discovery learning, alternative views, and reward user initiative. Ideally, it should encourage students to develop a capacity to abstract learning to other courses or knowledge domains. Boden (1991) sees agents in terms of conceptual spaces and creativity. She describes one potential use of agents as helping people to "map, explore, tweak, and (ultimately) transform conceptual spaces of many different kinds".

Workshops run during the design phase of the ASC project identified the need for the incorporation of an intelligent help system. Given the nature of the learning task, and the program's use of rather complex and arcane notation, a help system that could identify erroneous or irrelevant responses and offer contextual help was seen as essential to the project's success as an effective learning tool.

The workshops also highlighted a number of key questions about aspects of agent design and implementation. What scope of responses would the learning agent attempt to deal with? Would the agent be motivated to step in when an inappropriate response was detected, or would it only appear on request from a user? Having tracked the student's use of the software together with input, would it offer a review of progress to date? What levels of hinting would be appropriate?

Another design issue centred on look-and-feel aspects. That is, how should an agent manifest itself (and need this be a fixed mode)? Are there advantages in using an animated graphic representation with all the inherent design overhead, or would a simple text box suffice? Should the agent have a voice?

In order to resolve these queries we went back to the learning objectives, which were fairly straightforward:

• to become familiar with the soil classification system
• to learn to classify different soil types in the field
• to learn what soil properties govern particular soil types
• to learn how to interpret soil data and give them contextual meaning
• to gain experience in using the classification system with sample soil
• information, i.e., putting the theory into practice
• to gain experience in using the course text

A substantial literature review showed us that other researchers had grappled with many of these design issues and come up with apparently workable solutions. Murphy and McTear (1997) use a Learner Model, which stores learner characteristics that are relevant to the tutoring strategies employed by their computer aided language learning (CALL) system called CASTLE. The system's Tutoring Module contains a series of adaptive templates, allowing it to tailor representations of grammatical and linguistic concepts for each learner. It also has an inference engine, which provides the system with a current assessment of the user's proficiency so that the system knows when to intervene and offer remedial help, and what level of help to offer. Three hinting levels are available on demand from a user, each of which affects the user's subsequent proficiency level. Level 3 help, for example, supplies the correct answer, and lowers the proficiency level for that topic the most. Input errors are recorded as a user progresses through an exercise. Exceeding an error threshold (e.g. making three errors of the same category) causes the Tutoring Module to recommend a remedial exercise, based on the current proficiency level and the system's understanding of the required prerequisite knowledge. In addition, a combination of proficiency scores gives learners a measure of overall proficiency in the subject. This dynamic approach to measurement of a user's proficiency is employed by several intelligent tutor programs, enabling them to track progress and provide highly contextual help at the appropriate time.

The current prototype implementation of ALLIANCE adopts a rather simplified version of the coached problem solving approach used by the ANDES program to teach physics. As the knowledge domain is highly specialised, it was decided that the predictive analysis techniques (based on Bayesian networks) used by ANDES were not necessary to provide motivation for the agent software. The organised nature of the knowledge map and its logical sequencing of classification steps means that the software can make use of basic heuristics and logging techniques to determine simple facts about a user's proficiency and current level of completion. Users currently have total control over the agent software, which can be disabled at any time.

ALLIANCE is motivated to intervene when a user has made an attempt at classification that is incorrect. Given this input, together with its knowledge of what resource sites the user has already visited, and its own understanding of the prerequisite knowledge required for the current topic level, it will suggest a 'generic' site where the missing knowledge can be found and reviewed. The agent avoids directing or moving the user directly to the specific site where possible, usually making suggestions that use a general classification term, such as ' I think you should review the Physical Properties for this soil'. This allows students to discover fact-holding sites for themselves, and gives them practice in navigating the various resource categories.

The representation of the ALLIANCE agent as a graphic entity fulfils two needs. First, it accords well with Bates' (1994) notion of agents being 'believable' and therefore accepted as suitable tutorial assistants. Bates' Oz project explored the use of simple graphical agents driven by underlying programmed 'emotions'. Second, it fits within ASCOT's organisational approach of using HTML pages that combine resources such as a database of soil images with text-based courseware sections. While the ALLIANCE agent has a characteristic mode of expression, its 'emotions' are quite limited, being (presumably) inferred from its physical actions. We avoided choosing graphic forms that were too anthropomorphic, as users tend to imbue human-like agents with unrealistic abilities never intended or programmed by the system creators. The current prototype appears as an animated earthworm ('Al'), which can communicate with both speech-balloon text and text-to-speech output. As Al can be positioned dynamically anywhere on the page, gestural information (such as indicating specific soil components) can play a part in the interaction, enhancing or replacing text responses.

In its job of augmenting the learning experience, ALLIANCE undertakes a number of tasks such as:

• keeping a log of user interaction and navigation and providing updates of progress towards the learning goal
• providing feedback on student soil allocation responses
• directing a user to networked resources that are essential to completion of the learning task
• providing tutorial assistance and contextual help as inferred from a user's actions or requested by a user
• Planned extensions to the software will (ideally) include:
• modeling a user's interaction with the program and optimizing the interface to the user's benefit. For example, agent-user interaction could move from intervention to a request-only mode depending on the user's level of proficiency.
• learning, through user modeling, an individual's preferences and current level of attainment, then offering appropriate information on request

At the current level of development, ALLIANCE attempts to augment the stated learning objectives of the ASCOT tutorial program in what we believe is a realistic and realizable way. We have chosen a software architecture that can be served across any network, opening up possibilities for both distributed and collaborative learning modes. We anticipate that the experience gained in researching and implementing various ALLIANCE agent prototypes will translate to other disciplines and alternative modes of course delivery.

The effectiveness of learning agents such as ALLIANCE in offering useful (and acceptable) help to a student will be the subject of an evaluation phase of the authors' research into learning agents in general. The logging feature used by ALLIANCE offers students, educators and evaluators valuable feedback on how the tutorial program is being used, and how it might be improved.

Future developments in this area of artificial intelligence will likely concentrate on the use of different media for human-computer interaction. We suspect that given the choice, users will prefer to have tutorial assistance transmitted to them in aural form, leaving them free to concentrate on the subject matter at hand. Recent advances in continuous speech recognition may mean that a large part of assistance/response interactions are conducted as spoken conversations. This would be particularly pertinent in language learning settings. An ideal agent tutor would operate as transparently and unobtrusively as possible, motivated by a need to assist rather than judge a student's output.

(c) Tom Fenton-Kerr, Tony Koppi, Marcel Chaloupka, Stephen Cattle, Alison Anderson

The author(s) assign to ASCILITE and educational and non-profit institutions a non-exclusive licence to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. The author(s) also grant a non-exclusive licence to ASCILITE to publish this document in full on the World Wide Web and on CD-ROM and in printed form with the ASCILITE 97 conference papers, and for the documents to be published on mirrors on the World Wide Web. Any other usage is prohibited without the express permission of the authors.