EdTech'90 Proceedings: Hall - developing an intelligent tutoring system

Developing an intelligent tutoring system

This paper is concerned with the development of an Intelligent Tutoring System for use in teaching mathematics. Research in the field of intelligent tutoring systems is generally conducted by cognitive scientists or computer scientists whose main concern is the development of a system that emulates human behaviour, or at least a system that arrives at the same conclusions achieved by a human. This paper takes the view that fundamental to the development of intelligent tutoring systems is an understanding of processes related to pedagogy and learning. That is, this paper argues that the extent to which an intelligent tutoring system is educationally valuable partly depends on the ability of cognitive scientists and computer experts, but also depends on a detailed understanding of the teaching-learning process. In particular this paper examines the intricacies and complexities of teaching and learning seemingly simple mathematical procedures.

The issues

The first decision in developing an intelligent tutoring system is to select the content area. In the research reported here the topics were the addition and subtraction of whole numbers, and the addition of fractions. Educational objectives of the intelligent tutoring system are then set: in mathematics education there has been a long standing argument about teaching for understanding, and about procedural and conceptual knowledge. The intelligent tutoring system must take these views into account in its objectives and in its presentation. Developers of the intelligent tutoring system need then to examine research as to what is known about student learning in that field, including the problem of representing mathematical concepts using two and three dimensional objects, and must then investigate good teaching practice in the field. Each of these issues is analysed in later sections of this paper.

Understanding as an objective in mathematics teaching

Much research and development has taken place in recent years concerning the teaching and learning of mathematics. There is a long history of mathematics taught as a set of rules to be learned and practised. The rejection of this very traditional approach to mathematics also has a long history (Breslich, 1933; Durell, 1929; Taylor, 1938). Contemporary approaches to the teaching of mathematics see 'understanding' of mathematics by the learner as fundamental to the teaching process.

The term 'understanding' reflects at least two views. One view is that of understanding as 'remembering the step by step application of the appropriate rule to obtain the correct answer'. This has its basis in traditional classroom teaching, with the learner simply being a smaller version of an adult, who has to learn the rules of mathematics. A second view of 'understanding' in mathematics is that the learner is able to perform the step by step sequence to achieve the correct answer because he or she understands the concepts behind the rules and has a good knowledge of the relationship between these steps.

Procedural and conceptual knowledge

Procedural knowledge is concerned with following the appropriate sequence of steps to obtain a correct answer, whereas conceptual knowledge requires both an ability to obtain the correct answer and to be able to explain why the steps in the algorithm are appropriate (Hiebert, 1986). Learning procedures 'by heart' so as to obtain a correct answer without any deeper understanding is clearly rote learning, and involves only procedural knowledge. That is, procedural approaches involve choosing the correct rule and substituting known values so as to find the value of the unknown: success is heavily dependent on the context in which learning occurred.

Procedural knowledge is concerned with 'how do I get the correct answer?', and is often linked to 'skills', at the expense of understanding. It is seen by mathematics educators as the poor cousin of conceptual understanding. There is a long history of teachers arguing that by teaching 'the skills' a fuller understanding will occur at a later time. This may indeed be the case, but it rarely happens unless the teacher sets out consciously to provide that elaboration of the skill at a later date. In reality, for the great majority of school children, a fuller understanding of mathematical relationships does not occur through drill and practice alone.

Conceptual knowledge involves understanding relationships between concepts and procedures. A student with conceptual knowledge of a topic is more likely to be able to explain his or her solution to a problem, is more likely to have an individual approach to the solution to particular problem, and is more likely to be able to transfer this knowledge to related problems, than is a student who is limited to procedural knowledge. The learner with conceptual knowledge has a well developed conceptual map of the interrelatedness of mathematical ideas, and a well developed network of mathematical concepts. Of course, a student may have conceptual knowledge in some topics, and operate with procedural knowledge in other areas of mathematics.

Conceptual knowledge may be seen to exist at a number of levels. The primary level is where mathematical relationships are understood and can be applied effectively: for example, recognising the appropriate trigonometrical ratio to find an angle in a right angled triangle. Then there is a higher, more reflective level where one sees the relationship that exists between trigonometrical ratios and algebraic statements and manipulations in general. That is, at the higher level, the mathematical understandings are richer, and one can analyse problems to plan the most appropriate method of attack based on earlier experiences and examination of the likely range of possibilities. The essential element to conceptual understanding of mathematics, at both the primary and higher levels, is the understanding of the relationships between mathematical concepts, and the ability to be able to free the mathematical concepts and procedures from the context in which they were actually used; that is, to increase the generalisability of the concepts to new contexts. Von Glaserfeld (1987) argues that the more abstract the concepts and operations, the more reflective activity will be needed, that these operations and concepts are products of several levels of abstractions, and that successive acts of reflection are necessary.

The assumption is made implicitly, if not explicitly, that school mathematics should be about the conceptual understanding of mathematics. Procedural understandings are insufficient. Given the meaning of 'understanding' in contemporary mathematics education, it appears that learners must attain conceptual knowledge in the field of study. Conceptual knowledge is argued to be 'better' than procedural knowledge because it involves a higher level of understanding. In conceptual knowledge the problem solver is more likely to be able to solve a novel question through his or her more fully developed cognitive schema, and through a greater likelihood of being able to successfully transfer previous experience to new situations. Von Glaserfeld (1987) asserts that "the primary goal of mathematics instruction has to be the student's conscious understanding of what he or she is doing and why it is being done". Conceptual knowledge will allow the learner to organise his or her knowledge more effectively, and memory and recall will be easier, because the relationships between the mathematical concepts are understood.

If intelligent tutoring systems are to be developed for widespread classroom use in mathematics the argument of procedural versus conceptual have to be addressed. Indeed, the longer term value of such intelligent tutoring systems is likely to be enhanced if they are based on approaches more likely to lead to conceptual understandings than if they limit themselves to emphasising procedures.

The role of embodiments in mathematics learning

This use of representations in mathematics, especially through the widespread classroom use of manipulative materials, exemplifies one of the major aspects of modern mathematical pedagogy. This aspect is concerned with the representation of mathematical concepts and principles through two or three dimensional diagrams and structures. The main questions of interest to researchers include what representations are most effective in mathematics, ought these representations differ with the age of students being taught, and just how is it that learners use these representations to internalise the mathematical concepts involved so that they construct cognitively efficient inter-related conceptualisations of mathematical knowledge? The term most associated with these aspects of teaching and learning in today's literature is 'embodiments'.

The ways in which conceptual knowledge and procedural knowledge are developed is still the subject of research. It is not a simple issue. Many teachers, especially of elementary school mathematics, make use of teaching materials that embody mathematical ideas. The learner is meant to use this material initially to solve mathematical questions, and then in some way to internalise the mathematical concept embodied in the material. For example, by using Multi-based Arithmetic Blocks the learner is expected to internalise the concept of place value which causes so much difficulty in the performance of arithmetic calculations. The same materials are designed to help the learner understand addition and subtraction algorithms involving almost any size numbers. But just how the concept of place value or the structure of the algorithm is internalised is unclear. Little detailed understanding exists as to the use of embodiments and symbolisations in general (Kaput, 1987): in spite of research into different representations of a range of mathematical concepts there is no systematic agreed upon theory.

A fully developed understanding of a concept means that the student will be able to use manipulative materials and diagrams to solve problems about this concept, will be able to use spoken and written language to solve problems and describe their solutions, and will be able to apply this problem solving ability to the real world (Lesh et al., 1987): effective teaching will have to include all these kinds of experiences. This idea is supported by Booker et al. (1980) who argue that arithmetic activities ought not to be limited to written numerals, but ought also to include activities associated with the word name and activities involving concrete/ diagrammatic representations of numbers. This is clearly important for developers of intelligent tutoring systems.

Present day educational computer software available for use in mathematics sometimes uses diagrammatic representations, but relies mostly on written symbols (numerals) and seldom uses number names. Such software may use diagrammatic representations of manipulative materials, but the link to written or other symbolisms is rarely apparent. At the same time it must be realised that learners have to be given time to familiarise themselves with the various representations in order to be able to use them effectively, and to have a greater chance of internalising the concepts involved. For example, even calculators are a hindrance in solving arithmetic problems if the user is not totally familiar with the operation of the calculator (Cooper and Hall, 1989).

Dufour-Janvier et al. (1987) add further support to the earlier views expressed here, when they outline a range of motives for using 'embodiments', or as they called them 'external representations'. In their view representations are an inherent part of mathematics, they make possible multiple embodiments of a single concept, they help overcome specific difficulties and make mathematics more attractive and interesting.

With regard to intelligent tutoring systems then, there is a clear need to emphasise the flexibility with which such systems must deal with representations of mathematical concepts. This need for flexible representations is likely to increase in importance as the availability of mathematics education software, including intelligent tutoring systems, increases.

Teaching strategies

The importance of the way mathematical concepts are represented has been discussed, as have the kinds of knowledge learners' need to achieve. But there is still the issue of teaching strategies. That is, conceptual knowledge may be more desirable than procedural knowledge, and embodiments used in computer based teaching may have to be flexible, but this does not solve the problem of how topics should be taught.

The question of the most effective teaching style is clearly complex. The behaviourist style of teaching with its emphasis on stimulus-response connections is eliminated since mathematics educators generally regard it as unpopular and believe it is likely to lead to procedural rather than conceptual knowledge. According to Howson (1983) this approach has exerted little influence on mathematics teaching.

The Integrated-Environmentalist style (where mathematics teaching is environmentally based) and the Formative teaching style (based on Piagetian principles) would seem to be too complex in terms of design and construction for use in an intelligent tutoring system, and in any event there is little support for them in the schools (Howson, 1983). The Structuralist approach, emphasising the structure of mathematics, seems to have a good deal in common with the design and structure of an intelligent tutoring system. Howson notes that there has been little rigorous research into this teaching style, but argues that it is a useful approach. Indeed there is a good deal of support in the literature for the Structuralist approach to teaching mathematics (Booker et al., 1980; Bruner, 1963; Dienes, 1960, 1973).

A pedagogical model for mathematics teaching

In mathematics education the acquisition of conceptual knowledge is believed to be facilitated through the use of manipulative materials. 'Embodiments' are an important part of mathematical pedagogy (Booker et al., 1980; Davis, 1986; Hall, 1981a, 1981b, 1982). More recently there has been a closer examination of these assumptions so as to understand how learning actually takes place, how concepts are formed and organised, how efficient cognitive networks are developed, and how concepts can best be stored and retrieved. In developing an intelligent tutoring system it must be remembered that we are primarily concerned with the view of 'embodiments' as a teaching methodology: but this more recent research emphasis into how learning actually occurs through the use of 'embodiments' provides us with a model upon which to base pedagogy.

Ohlsson and Hall (1990) have developed a model that predicts that the pedagogical effectiveness of an embodiment is a function of four variables: the ease with which the learner can understand the description of the embodiment procedure, the ease with which that description can be proceduralised, the degree of isomorphism between the embodiment procedure and the intermediate or target procedure, and the degree to which the transformation of the embodiment procedure into the intermediate or target procedure can be done through simplifications.

The model focuses on the isomorphism between the embodiment procedure and the intermediate or final target procedure. This model proceeds in four steps:

Construct an information processing model of the embodiment procedure as well as the target procedure.
Run the models to generate expanded traces.
Decide which entries in the expanded traces correspond to each other.
Calculate the isomorphism index.

Figure 1 shows an expanded trace for the use of Multi-based Arithmetic Blocks in solving an addition of two three digit numbers. In the figure F, L and U represent Flats, Longs and Units which are the names given to the materials to emphasise the place values hundreds, tens and units.

Figure 1: The procedure for addition with MAB blocks.


0.	Add (5F, 3L, 2U) and (1F, 4L, 9U).
1.	Process the units (2U, 9U).
	1.1	Join 2U to 9U => 11U.
	1.2	Trade 11 U.
		1.2.1	Decompose 11U.
			1.2.1.1	Count out 10 units.
			1.2.1.2	Move remaining 1 unit down.
		1.2.2	Exchange 10 units.
			1.2.2.1	Move 10 units to the bank.
			1.2.2.2	Move 1 long to the L column.
2.	Process the longs (3L, 4L, 1L).
	2.1	Join 3L to 4L => 7L.
	2.2	Join 7L to 1L => 8L.
	2.3	Move 8L down.
3.	Process the flats (5F, 1F).
	3.1	Join 5Fto 1F => 6F
	3.2	Move 6F down
Total number of entities: 17.

There are seventeen steps that the learner must take in order to answer this question. Some of the steps are physical actions but others are thought processes. Next another trace is devised, for example, a trace of an 'expert' in adding three digit numbers: this is the target procedure. The steps in the embodiment procedure are compared with the steps in the target procedure. A measure of the isomorphism between the two procedures is calculated using a formula that takes into account the number of steps in each trace, and the number of steps in one trace not matched by steps in the other trace. Ohlsson and Hall (1990) maintain that the higher the isomorphism as calculated using the formula, the more effective will be the use of the embodiment in teaching the target behaviour. Put another way, the higher the measure of isomorphism, the greater will be the achievement scores of learners. This formula was successfully applied to three pieces of research, two of which were computer based.

To generate sets of production rules for these procedures takes a matter of hours. The ease with which the measure can be applied makes it possible to generate hypothetical designs, calculate how well they come out on the isomorphism index, revise them, calculate the index again. The model claims that the degree of isomorphism is one of four variables that determine the effectiveness of an embodiment. It is not the sole determinant, but the researchers suggest that it is safe to predict that if the isomorphism is low, then the embodiment will not be effective.

According to this model the teacher has three tasks. First, to describe the embodiment procedure in such a way that the learner can proceduralise it. Second, to specify the analogical mapping between the embodiment procedure and the intermediate procedure. Third, to guide the learner through the successive simplifications of the intermediate procedure until it coincides with the target procedure. In the context of intelligent tutoring systems, the designers of such systems may find this model an appropriate base from which to develop specifications for the system. In any event the model proposed by Ohlsson and Hall clearly shows the complexity of having students learn mathematical concepts and skills, and indicates that specifying the teaching component of an intelligent tutoring system will itself be a complex problem.

Conclusion

Inherent in this paper is the view that intelligent tutoring systems will lessen their potential impact if their designers believe that knowledge is a commodity, directly transferable from one person to another in the way that everyday items are bought and sold. The software's role is not to dispense knowledge in accurately sized dollops, but to provide the opportunity for the learner to come to a correct and efficient conceptual organisation. This paper has argued that in the world of classrooms, education and training the development of an intelligent tutoring system is indeed problematic.

The Ohlson and Hall model summarised here addresses the major issues raised in this paper. The model describes a method for using embodiments in both face to face and computer based teaching, it provides a pedagogy based on a well documented view of learning and clearly demonstrates a method of assisting a learner towards both expert behaviour and an understanding of mathematical concepts. The model provides a useful tool suitable for the design of the teaching component of intelligent tutoring systems.

References

Booker, G., Irons, C. and Jones, G. (1980). Fostering Arithmetic in the Primary School. Canberra: Curriculum Development Centre.

Breslich, E. R. (1933). Secondary School Mathematics and Changing Curriculum. Math T, 26, 327-9.

Bruner, J. S. (1963). The Process of Education. New York Vintage Books.

Cooper, T. and Hall, N. (1989). The effect of instruction on calculator usage on performance in multi-step word problems with calculators. Research monograph, Brisbane CAE.

Davis, R. B. (1986). Conceptual and Procedural Knowledge in Mathematics: A Summary Analysis. In J. Hiebert (Ed), Conceptual and Procedural Knowledge: the Case of Mathematics. Hillsdale, NJ: Lawrence Erlbaum Associates.

Dienes, Z. P. (1960). Building Up Mathematics. London: Hutchinson.

Dienes, Z. P. (1973). The Six Stages in the Process of Learning Mathematics (P. L. Seaborne, Trans.). Paris: OCDL.

Dufour-Janvier, B. et al. (1987). Pedagogical Considerations Concerning the Problem of Representation, in C. Janvier (Ed), Problems of Representation in the Teaching and Learning of Mathematics. Hillsdale, NJ: Lawrence Erlbaum Associates.

Durell, F. (1929). Making the concrete and abstract help each other in mathematics. Sch Sci Math, 29, 702-713.

Hall, N. (1981a). Teaching Decimals - Diagnosis and Remediation. In A. Rogerson (Ed), Mathematics: Myths and Realities. Melbourne: MAV.

Hall, N. (1981b). Teaching Fractions - Diagnosis and Remediation. In A. Rogerson (Ed), Mathematics: Myths and Realities. Melbourne: MAV.

Hall, N. (1982). Towards a Theory of Teaching Mathematics. In J. Veness (Ed), Mathematics - A Universal Language. Sydney: AAMT.

Hiebert, J. (Ed) (1986). Conceptual and Procedural Knowledge: the Case of Mathematics. Hillsdale, NJ: Lawrence Erlbaum Associates.

Howson, A. G. (1983). Curriculum Development and Curriculum Research: Part C. Windsor, UK: NFER-Nelson.

Kaput, J. J. (1987). Representation Systems and Mathematics, in C. Janvier (Ed), Problems of Representation in the Teaching and Learning of Mathematics. Hillsdale, NJ: Lawrence Erlbaum Associates.

Lesh, R., Post, T. and Behr, M. (1987). Representations and Translations Among Representations in Mathematics Learning and Problem Solving, in C. Janvier (Ed), Problems of Representation in the Teaching and Learning of Mathematics. Hillsdale, NJ: Lawrence Erlbaum Associates.

Ohlsson, S. and Hall, N. (1990). The Cognitive Function of Embodiments in Mathematics Instruction. Technical Report No. KUL-90-02. Pittsburgh: Learning Research and Development Centre, University of Pittsburgh.

Taylor, E. H. (1938). Mathematics for a Four Year Course for Teachers in the Elementary School. Sch Sci Math, 38, 499-503.

Von Glaserfeld, E. (1987). Learning as a Constructive Activity. In Janvier, C. (Ed), Problems of Representation in the Teaching and Learning of Mathematics. Hillsdale, NJ: Lawrence Erlbaum Associates.

Author: Neil Hall, School of Policy and Technology Studies in Education, The University of Wollongong, Wollongong NSW.
Please cite as: Hall, N. (1990). Developing an intelligent tutoring system. In J. G. Hedberg, J. Steele and M. Mooney (Eds), Converging Technologies: Selected papers from EdTech'90, 186-193. Canberra: AJET Publications. http://www.aset.org.au/confs/edtech90/hall.html

[ EdTech'90 contents ] [ EdTech Confs ] [ ASET home ]
This URL: http://www.aset.org.au/confs/edtech90/hall.html
© 1990 The author and ASET. Last revised 13 May 2003. HTML editor: Roger Atkinson
Previous URL 21 Aug 1998 to 30 Sep 2002: http://cleo.murdoch.edu.au/aset/confs/edtech90/hall.html
Created 21 Aug 98. Last update: 21 Aug 98. HTML editor: Roger Atkinson [atkinson@cleo.murdoch.edu.au]