Jan Elen
Centre for Instructional Psychology and Technology
University of Leuven, Belgium
jan.elen@ped.kuleuven.be

L. Philip Louw
Department of Curriculum Studies
University of the Free State, South Africa
louwlp.hum@mail.uovs.ac.za

Key Words: adjunct aids, textbooks, log files

Abstract

Adjunct aids are instructional interventions inserted in textbooks in view of supporting learners to process the information. Different types of adjunct aids are assumed to support different cognitive processes. Research on adjunct aids has focussed on the learning effects of single types of adjunct aids. In this study, the learning effects of combinations of adjunct aids are studied. After completing a pre-test, 255 students studied a text presented on 21 separate computer screens in one of eight conditions. Their use of the adjunct aids was logged. After studying the text, a post-test was administered. Results show no impact of conditions on either knowledge or transfer items. However, when the use of the adjunct aids is considered, it is shown that this result can be explained by referring to the use of the aids at least for knowledge items.

Introduction

Adjunct aids are instructional interventions inserted in textbooks in view of supporting learners to process the information. Different types of adjunct aids are assumed to support different cognitive processes. Research on the effects of adjunct aids has flourished for the fourty years (for overviews, see: Rothkopf, 1996; Grabowski, 2004). This research has largely been part of the attempts to better understand how learners process text and how text comprehension can be promoted (e.g., Van Dijk & Kintsch, 1983). Having a cognitive psychological orientation, research on adjunct aids has contributed to elaborating information-processing models and to identifying implications of such models for learning and teaching. Results on retention and transfer tests, for instance, have been used as indicators of the kind of cognitive processing learners engage in (Mayer, 1996). Given its cognitive orientation, research on adjunct aids has been carried out in highly controlled settings. In most cases the learning effects of one particular type of adjunct aids and of fixed numbers of adjunct aids have been studied. Prototypical are the studies on advanced organizers (e.g., Mayer, 1979) or inserted questions (e.g., André, 1979). While this approach may help to increase internal validity, it may at the same time reduce the ecological validity of these studies. The results may not apply to actual instructional settings in which designers and developers of instructional texts insert multiple and different types of adjunct aids in their materials. In view of increasing the ecological validity of research on adjunct aids, this study investigated the learning effects of combinations of adjunct aids. Furthermore, it is essential to study multiple aids in order to identify possible interaction effects.

Research on adjunct aids in textbooks commonly assesses the learning effects by analysing the results of a post-test without analysing the actual use of the adjunct aid by the learners. This is a problem because multiple studies on instructional interventions indicate that learners often do not or not as intended use instructional interventions (Clarebout & Elen, in press). In order to deal with this problem, this study logged the students’ use of the adjunct aids when studying a text on a computer screen.

The main question in this study addresses the impact of combinations and varying numbers of aids on the use and the learning effects of adjunct aids in textbooks. In addition, it is studied how the use of adjunct aids affects learning outcomes.

In order to answer these questions, students were asked to study a text that was presented on a computer screen. In the experimental conditions (combinations of) three types of adjunct aids (questions, figures, and examples) were inserted in the text, whereas the text in the control condition contained no adjunct aids. Examples, questions and figures were selected because they clearly differ in nature and have been argued to trigger different cognitive functions. In line with the selecting, organizing, integrating (SOI) model, Mayer (1996, 1999) has suggested that different instructional interventions may support different categories of learning activities. Adjunct questions have been argued to support learners to focus on the relevant information (selecting); figures have been argued to support the organization of incoming materials (organizing), while examples as well as illustrations were suggested to help learners to activate prior knowledge and integrate new information in prior knowledge (integrating). Although these aids were added to have a specific mathemagenic or learning-generating effect (Rothkopf, 1970), the absence of a one-to-one relationship between an adjunct aid and a particular cognitive function is acknowledged (Kealy, Bakriwala, & Sheridan, 2003).

This contribution discusses the empirical study after having briefly reviewed the research on the effects of the three aforementioned adjunct aids and the actual use of aids by learners.

Adjunct Aids

Questions

Inserted questions have attracted ample research attention (e.g., André, 1987). This research shows that questions have mainly a selective and a restructuring function. Research commonly shows that readers are better able to recall topics targeted by pre-questions (Kealy et al., 2003). In the case of post-questions a broader generative effect was found. Post-questions may encourage readers to mentally review the material.

Figures

In various studies the use of figures in addition to textual information has been proven to be highly valuable (for a review: Anglin, Vaez, & Cunningham, 2004). Mayer (2001) called this the multimedia effect. Levie and Lentz (1982) identified four functions for graphics/pictures in text: an attentional, an affective, a cognitive, and a compensatory function. While initially Paivio’s dual coding theory (Clark & Paivio, 1991) induced widespread optimism about the potential of pictures, recent research has induced more caution. In line with their integrated model of text and picture comprehension, Schnotz and Bannert (2003) for instance, concluded that only task-appropriate graphics may support learning, whereas task-inappropriate pictures or graphics may induce mathematantic effects. In the latter case a decrease rather than an increase in learning is observed.

Examples

Examples are so-called explanatory adjunct aids (Elen, 1995). They present additional information that may help the reader to better understand the text. Increased text comprehension results from helping the learner to connect the information in the text with the reader’s prior knowledge (Twohig, 1982). This is assumed to help the integration of the new information. While in general positive effects of examples can be expected, research shows that examples have to meet criteria such as understandability and relevance (Elen & Lowyck, 2000). Furthermore, research on seductive details (Harp & Mayer, 1998) in which interesting but less relevant illustrations are added to the information, also reveals that adding highly interesting information for instance in examples, might be detrimental for learning.

Use of Aids

Research on adjunct aids has mainly been done in (quasi-) experimental settings. Various researchers have pointed out that the use of aids in such settings may differ from use in more ecological settings. More specifically, studies show that regularly adjunct aids are only seldom used as intended by the text designers. This is the case when instructional interventions such as adjunct aids are added to the information (Clarebout & Elen, in press) as well as when embedded instructional interventions are provided (Martens, Valcke, Poelmans, & Daal, 1996)

Research Questions

Considering the outcomes of previous research on adjunct aids, this study investigated the effect of combinations and varying numbers of adjunct aids on learning outcomes (knowledge and transfer). Because previous research explains the lack of effects by referring to the minimal use of adjunct aids, the use of the aids was also analysed. The relationship between the use of aids and their learning effect was studied.

Research on combinations of adjunct aids is largely missing. Hence, there is only a limited basis for formulating hypotheses. Nevertheless, it is hypothesized that an effect of adjunct aids can be established and that the effect will be additive on the one hand and related to use of the aids on the other. The additivity hypothesis implies that an effect is expected for learning outcomes that are immediately related to a specific adjunct aid and that the overall effects of adjunct aids become larger if more and different types of adjunct aids are made available. The use-hypothesis specifies that effects will only be found when the aids are actually used.

In short, the study aims at answering two basic research questions. What are the learning effects of combinations and varying numbers of adjunct aids, and what is the relationship between the use of adjunct aids and learning outcomes?

Method

Participants and Design

Research participants were 255 (221 female, 34 not specified) students attending undergraduate classes at a public university in the middle of South Africa. All students were majoring in education and volunteered to participate in the study. Students could gain a limited number of extra points by participating in the study.

All respondents were randomly assigned to one out of eight conditions in a pre-test post-test experimental study with number of aids (five, ten or fifteen), type of aids (questions, examples or figures), and use of the aids as independent variables and learning outcomes (retention and transfer) as dependent variables.

All students studied an instructional text on a computer screen. The eight conditions differ with respect to (a) the number of adjunct aids inserted (0, 5, 10, or 15 adjunct aids), and (b) the nature of the adjunct aids inserted (examples, questions, or figures) (see Table 1). In a first condition, the control condition, no adjunct aids were added. In conditions 2, 3, and 4, five adjunct aids were inserted: five examples in condition 2, five questions in condition 3 and five figures in condition 4. The instructional text in conditions 5, 6 and 7 contained 10 adjunct aids with in each condition two sets of five adjunct aids. In condition 5, 5 examples and 5 questions were inserted; in condition 6, 5 examples and 5 figures, and in condition 7, 5 figures and 5 questions. Finally, in condition 8, 15 adjunct aids were inserted in the instructional text: 5 examples, 5 questions, and 5 figures.

Table 1 Control and experimental conditions

Materials

Instructional text

The instructional text was a 6446 word passage (7182 words in the version with all three types of adjunct aids) adapted from a chapter on technology management by Newby, Stepich, Lehman and Russell (1996). The different versions of the computerised texts were developed using Macromedia Director 8.5. This text was selected by the teacher of this group of future teachers as being appropriate in level and also relevant given recent plans of the South-African department of education. Distributed over 21 screens, the text discusses various issues with respect to the integration of ICT in schools. In a first section the need for a technology plan is discussed. Hardware and software issues are discussed in the second and third section, while the final section describes personnel issues. On the last screen, a summary concludes the text.

Figure 1. Example of computer screen and related adjunct aid

Adjunct Aids

Adjunct aids gained access to the instructional text by clicking on a labelled button. The integration of the adjunct aids is based on the following considerations. In order to ensure balance, adjunct aids were evenly spread over the 21 screens with a maximum of 1 adjunct aid per screen (e.g., two related screens are displayed in Figure 1). This means that in condition 8 with 15 aids, there were 15 computer screens with an aid and 6 screens without an aid. In order to ensure comparability of the eight conditions and to avoid interfering sequence effects, an adjunct aid was always linked to the same computer screen and different types of adjunct aids were inserted in the same sequence. These considerations resulted in a distribution of the adjunct aids as outlined in Table 2.

Table 2 Distribution of adjunct aids over the text

Questions are inserted in view of supporting information selection activities of students. Questions triggered students to think more deeply about an information element or to link a specific element to their person. In the conditions with inserted questions (conditions 3, 5, 7, and 8) students could access five inserted questions. The following question, for instance, was added to the screen in which different LAN-configurations were presented: “If your principal would ask your advice on the installation of a local-area-network (LAN), what would be your arguments in favour of a client-server model?”.

Figure 2. Figure added to text explaining computer lab lay-outs.

Cluster

As suggested by the SOI-model, figures can have two main functions (Mayer, 1999). If a figure is more a graphic representation, organization processes are supported, while integrating processes may get supported when the main purpose of the figure is illustrative. In the conditions with figures (conditions 4, 6, 7, and 8) students could access five figures by clicking on the figure button. None of the figures was purely decorative. Three figures illustrated the information on the computer screen by repeating part of the textual information in a pictorial way (e.g., see Figure 2), whereas two others highlighted in a diagram the structure of the text on the screen that contained the figure button (e.g., see Figure 3).

Figure 3. Figure added to screen discussing technology implementation plans.

Examples may support learners to connect their own prior knowledge with the new information. In the examples-conditions (conditions 2, 5, 6, and 8) students could access five examples that further contextualized the more general information in the text. Figure 1 shows an instance of an example in which a concrete software evaluation form is presented. The examples were constructed by selecting the most difficult element in the text. For instance, an excerpt from a mission statement was added to explain the meaning of the concept 'mission statement'.

Pre- and Post-test

Two parallel tests were constructed. One was randomly selected and used as the pre-test. The other served as the post-test. Each test contained 8 items. Four questions were recall items. The information can be immediately retrieved in the text. The four remaining questions were transfer items. In order to generate an answer, students had to use and transform the information in the text. For instance, the text discusses considerations to be made while selecting hardware for classroom use. In the transfer item, it was asked what can go wrong when selecting hardware for classroom use.

In order to analyze whether adjunct aids have a specific or a more general effect, some items directly related to an aid and others not. There was a clear relationship between one of the figures and two knowledge items in each test (see Figure 4), one example and one transfer item in each test, and one question and one transfer item in each test. For instance, in one of the inserted questions students are encouraged to think about the implications of internet use although this is not explicitly discussed in the text. One transfer question addressed the use of the internet.

Figure 4. Example of figure-related questions in pre- and post-test

1. You see here a figure of a classroom arrangement. Circle the label that rightly describes this arrangement.

1. Peripheral

2. Back-to-back

3. Classroom

4. Cluster

1. You see here a figure of a classroom arrangement. Circle the label that rightly describes this arrangement.

1. Peripheral

2. Back-to-back

3. Classroom

4. Cluster

Log-files

All the actions of the participants were logged in an Access database for further analysis. The file contains an identification number for each respondent and information on whether and at what time a text screen or aid is accessed by the respondent. Data from the log files were used to calculate the number of times (specific types of) adjunct aids were used, the time spent on working with the aids and the proportion of the total study time devoted to the adjunct aids. Based on the data in the database different aspects of use could be calculated both for each adjunct aid separately and all aids inserted in the text: the number of times adjunct aids were accessed; the duration adjunct aids were looked at; and the proportion of the total studying time attention was devoted to adjunct aid(s).

Procedure

All participants were present in two sessions. The pre-test was administered during the first session. Respondents could take as much time as they needed to complete it. Administration of the pre-test took about ten to fifteen minutes. During the second session one week later, students in groups of maximum thirty at a time studied in a dedicated computer room the instructional text presented on an individual computer screen in one of the eight conditions. On an individual basis, participants were randomly assigned to one of the conditions (see Table 1). Each version presented participants with the instructional text and a different combination of adjunct aids. After having studied the text and after having completed a non-related intermediate task in about 5 to 10 minutes, a post-test was administered containing items aiming at assessing knowledge and transfer. The total session took about 70 minutes. Five minutes were used for explaining the procedures during the session and how to handle the text on the screen. For this purpose a standardised PowerPoint presentation was shown. Students were told that they could study at their own pace and that they would be informed when 40 minutes had passed. After indicating that they had completed the study of the text, students received a non-related intermediate task. Completing the post-test took between fifteen and twenty minutes.

Data Analyses

As a first step in the analysis a paired t-test was done, to see whether students actually learned something. After having investigated the role of prior knowledge, the analysis focused on answering the two research questions. The first research question in this study relates to the impact of type and number of adjunct aid on learning outcomes. In order to answer this question one-way ANOVAs were done. Partial eta-squares were calculated to assess the effect-size. In line with Cohen (1988) an eta²-value between .01 and .06 is regarded as small to average, between .06 and .14 as average to large and above .14 as large to very large. The question on the impact of the use of aids on learning outcomes was also addressed though means of ANOVAs by taking the frequency of adjunct aids access, the absolute and relative time devoted to adjunct aids as independent variables and results on the post-test as dependent variables. For each indicator of use, three groups of students (low, middle, high) were created based on the percentiles. For analyses with respect to the use of adjunct aids, condition 1 (the control condition) was excluded given the absence of adjunct aids in this condition.

All analyses were done with the statistical package SPSS and an alpha level of .05 was used as significance level for all statistical tests. In all cases DUNCAN was selected as the post-hoc test.

Results

Scoring

Scoring of pre- and post test

Pre-tests and post-tests were scored by awarding one point for each correct answer and one point for each relevant element in the open questions. The maximum number of points for the knowledge part in each test was 4 points. Given the way the transfer questions were scored, no maximum could be defined a priori. Table 3 gives an overview of the results.

While the tests were conceptually completely parallel, paired samples t-tests reveal clear differences between results on the (different parts of) the pre- and post-test.

Table 3: Results on pre- and post-test

Scoring of log-files

In order to get a good understanding of the use of the adjunct aids, the log-files were analysed to reveal the above mentioned indicators of the use of the adjunct aids: (a) the total number of times the adjunct aids were accessed, (b) the total number of times each specific type of adjunct aid was accessed, (c) the total time the adjunct aids were studied, (d) the time each specific type of adjunct aid was studied, (e) the proportion of the total study time spend on studying the adjunct aids, and (f) the proportion of the total study time spend on each type of adjunct aid. Table 4 summarises the main results of all the conditions.

Table 4: Summary of log-file data

Learning outcomes

The initial analysis of the data has already revealed a clear difference between the results on the (different parts of the) pre- and the post-test. As a first step, the impact of prior knowledge on learning outcomes was investigated through means of one-way ANOVAs. Significant effects (see Table 5) with average to large effect-sizes were found for results on the pre-test as a whole, and on both the knowledge part and the transfer part. For the test as a whole a DUNCAN post-hoc test indicates that students with limited prior knowledge perform worse than students in the middle and high performing groups, whereas students in the middle group do also perform worse than students scoring high on the pre-test. For the knowledge part, DUNCAN highlights that students with low scores on the pre-test perform worse on the post-test than students from the middle- or high-scoring group. For the transfer part, a DUNCAN post-hoc test reveals the low performing group to score worse than the group with high scores on the pre-test. For the knowledge test, only marginally significant results were found.

Table 5: Impact of pre-test results on post-test results

The first actual research question pertains to the effect of different conditions on the learning outcomes. ANOVAs with condition as independent and the different aspect of the learning outcomes as dependent variables, reveal no effect of condition; for neither the post-test as a whole nor the knowledge- and transfer-parts separately. Only for two specific and similar items (items B and C: see Figure 4) were effects with average effect sizes were detected. Both items were linked to one specific aid, namely a picture that graphically clarified the concepts. A DUNCAN post-hoc test for the effect on the B item (F= (2; 248)= 2.141; partial eta² less than or equal to .06) reveals a difference between the questions condition on the one hand and the figures-questions, the figures-examples-questions, the figures-examples, and the figures conditions on the other. A DUNCAN post-hoc test for the effect on the C item (F(7; 254) = 2.074; eta² less than or equal to .06) again reveals lower results for students in the questions-condition than for those in the figures- and the figures-examples conditions. Moreover, an additional difference was found between the examples-questions condition and the no-aids condition on the one hand and the figures-examples condition on the other.

The impact of each adjunct aid was further investigated by looking for effects of respectively examples and no-examples conditions, questions and no-questions conditions and figures and no-figures conditions on related items. This may help to see whether the effect is specific or not. The analysis revealed an impact for the B item of the figures conditions (F(1; 224) = 6.110; eta² less than or equal to .03) indicating a better result of the figures conditions over the no-figures conditions. For the C- item, an effect is found for the questions-conditions (F(1; 223) = 6.363; eta² less than or equal to .03) showing that students in the no-questions conditions perform better than those in the questions-conditions. In both cases, effect sizes are rather small.

A similar analysis was done to see whether the number of adjunct aids (5, 10 or 15) affected the learning outcomes. No impact of number of aids on learning outcomes was found.

Use of adjunct aids

A second research question pertains to the relationship between the use of the adjunct aids and the learning outcomes. For all three independent variables, effects with rather small effect sizes were found on the knowledge questions a. For the effect of the access of adjunct aids (F (2; 205) = 3.561; eta² less than or equal to .04) the DUNCAN post-hoc test reveals a better result on the knowledge part for those students that used the aids most frequently in contrast to students who used them least frequently. Exactly the same results are found for the effect of proportion of time devoted to the adjunct aids (F(2; 205) = 3.879; eta² less than or equal to .04). For the effect of the time spent on the adjunct aids (F(2; 205) = 4.529; eta² less than or equal to .04) a DUNCAN post hoc reveals a difference between the students that spent most time and both the lowest and the middle group.

Finally it was found that the total study time did not affect learning outcomes.

Discussion

The results clearly show that students learned while studying the text. Their performance on the different parts of the post-test is significantly higher than their performance on the different parts of the pre-test. Furthermore, an effect of the pre-test could be revealed, which of course simply reconfirms the importance of prior knowledge.

In spite of obvious learning outcomes, the hypotheses themselves were not validated. Inserting more adjunct aids does not result in better learning outcomes. Moreover, increased use of adjunct aids only has a limited effect on the knowledge part of the post-test.

With respect to the impact of particular types of adjunct aids, results are less equivocal. Overall, an impact of adjunct aids is observed only in very specific cases. The inclusion of examples does not appear to contribute to learning. This might be due to the nature of the examples included. They intentionally illustrated the content in the text. For instance, the text argues for the need for a clear evaluation sheet for software evaluation. The related example presents such a sheet. For the questions conditions, an effect is found for only one specific item (the C item in figure 1). To attribute this to the specificity effect reported in the literature is a possibility. Its limited occurrence, however, calls for necessary caution. Similarly, an effect of figures is found for only one item (the B-item in figure 1). Clearly the aid may have helped simply because it contained the same figure that was used in the post-test.

The overall absence of an impact of adjunct aids is further confirmed by the absence of any difference for conditions with different numbers of aids. In this respect, however, further research must consider the impact of the amount of aids independent from the study of interactions between types of aids. In this study both variables are not completely independent as more aids also implies that combinations of aids were inserted.

It must to be concluded that the hypotheses proved wrong. There is to some extent a specific effect of adjunct aids but overall there is no effect and certainly no additivity effect. Such results are not uncommon and a lack of such an effect is commonly attributed to under-use or inadequate use of the aids by the learners (e.g., Peeck, 1993). That is exactly the reason why in this study the actual use of the aids was logged. In this respect, results show a clear effect of the use of adjunct aids on the knowledge items in the post-test. In other words, students who accessed the aids more frequently, spent more time consulting the aids and devoted relatively more study time to them, did better on the knowledge items. This is in line with expectations., Adjunct aids can have an effect, only when actually used by the students. However, the results show no such effect for the transfer items. Three reasons may account for this lack of effect. First transfer questions might have been far too difficult. Second adjunct aids may have been insufficiently functional to promote transfer. In terms of the SOI-model of Mayer, adjunct aids may have promoted selection and organisation but not integration. Of course, an adjunct aid can only promote transfer when functional in that respect. And third, the literature clearly shows that transfer-related learning outcomes require sufficient study time. It might be that generally, students simply did not study long enough to be able to answer the transfer questions.

Conclusions

In this study an attempt was made to acquire better insights in the interaction between adjunct aids. It was hypothesised that effects of adjunct aids would be additive and related to their use. In other words it was expected that adding adjunct aids would improve learning outcomes while recognizing that the use of the aids is a condition for any effect. This study hardly provides data that support these hypotheses. The study clearly reveals that students with higher prior knowledge do better on the post-test. It also indicates that students hardly use the aids. An effect of the aids is only found when they are actually used. Finally, the study seems to suggest that there is an effect of adjunct aids only when the information in the adjunct aids is directly functional to answering and item in the post-test. Although much care was taken in this study to ensure relevance of the adjunct aids, it is clear that in further studies the relationship between the adjunct aids and the post-test needs even more attention. More specifically, an attempt should be made to support all processes of the SOI-model (Mayer, 1996, 1999). This study induces the conclusions that adjunct aids are only beneficial if adapted to the learner, used by the learner, relevant to the task, and transparently linked to specific items in the post-test.

The results of this study can be interpreted in different ways. First, it reconfirms the importance of prior knowledge. Second, it reconfirms the need for calibration between the learner on the one hand, and the (textbook) designer on the other in order for the learning environment to be effective (Winne, in press). It is up to the designer to ensure that the adjunct aid is linked to the task and to the learner; it is up to the learner to grasp the functionality of the aid and use it appropriately. A well-designed adjunct aid may have mathemagenic effects as Rothkopf has repeatedly argued (1970, 1996). However, this study illustrates that such a learning-generating effect requires learners to use the aids. The study also suggest that adjunct aids may have mathemantic or counter-productive effects (Clark, 1988) if inadequately used by the learner. A well-designed aid is not sufficient; it needs to be adequately used by a knowledgeable learner. This reveals a clear limitation of this study. Further research on the use and effects of adjunct aids should not only register quantitative but also qualitative aspects of their use. Furthermore, future research should attempt to assess how knowledgeable learners are about instructional interventions (Lowyck, Elen, & Clarebout, in press).

Given all these results, the need for a renewed research effort on the effect of combinations of instructional interventions is clear. More particularly, in-depth investigations are needed on factors that determine whether and how such interventions are used.

References

André, T. (1979). Does answering higher level questions while reading facilitate productive learning? Review of Educational Research, 49, 280-318

André, T. (1987). Questions and learning from reading. Questioning Exchange, 1 (1), 47-86.

Anglin, G.J., Vaez, H., & Cunningham, K.L. (2004). Visual representations and Learning: The role of static and animated graphics. In D.H. Jonassen (Ed.), Handbook o Research on Educational Communications and Technology. Second Edition. Mahwah, NJ: Lawrence Erlbaum Associates.

Ausubel, D. (1960). The use of advance organizers in the lLearning and retention of meaningful verbal material. Journal of Educational Psychology, 51, 267-272.

Clarebout, G., Elen, J. (in press). Tool use in computer-based learning environments: towards a research framework. Computers in Human Behavior.

Clarebout, G., ,Elen, J., & Sarfo, F.K. (submitted). Measuring instructional conceptions with the ICON-questionnaire. Educational Research.

Clark, R.E. (1988). When teaching kills learning: Research on mathemathantics. In H. Mandl, E. De Corte, N. Bennett, H.F. Friedrich (Eds.), Learning and instruction: European research in an international context. (Vol. 2.2, pp. 1-22). Oxford, England: Pergamon.

Clark, J.M., & Paivio, A. (1991). Dual coding theory and education. Educational Psychology Review, 71, 64-73.

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Earlbaum Associates.

Elen, J. (1995). Blocks on the Road to Instructional Design Prescriptions: A methodology for I.D.-research exemplified. Leuven: Leuven University Press.

Elen, J., & Lowyck, J. (2000). Instructional metacognitive knowledge: A qualitative study of conceptions of freshmen about instruction. Journal of Curriculum Studies, 32(3), 421-444.

Harp, S.F., & Mayer, R.E. (1998). How seductive details do their damage: A theory of cognitive interest in science learning. Journal of Educational Psychology, 90 (3), 414-434.

Grabowski, B. L. (2004). Generative learning contributions to the design of instruction and learning. In D. H. Jonassen (Ed.), Handbook of research on educational communications and technology. Second Edition (pp. 719-743). Mahwah, N.J.: Lawrence Erlbaum.

Kealy, W. A., Bakriwala, D. J., & Sheridan, Ph. B. (2003). When tactics collide: Counter effects between and adjunct map and prequestions. Educational Technology Research & Development, 51, 17-39.

Klausmeier, H.J. (1980). Learning and Teaching Concepts. New York: Academic Press.

Levie, W.H., & Lentz, R. (1982). Effects of text illustrations: A review of research. Educational Communications and Technology Journal, 30(4), 195-232.

Lowyck, J., Elen, J., & Clarebout, G. (in press). Instructional conceptions: analysis from an instructional design perspective. International Journal of Educational Research.

Martens, R., Valcke, M., Poelmans, P., & Daal, M. (1996). Functions, use and effects of embedded support devices in printed distance learning materials. Learning and Instruction, 6, 77-93.

Mayer, R. E. (1979). Can advance organizers influence meaningful learning? Review of Education Research, 49 (2), 371-383.

Mayer, R. E. (1996). Learning strategies for making sense out of expository text: The SOI model for guiding three cognitive processes in knowledge construction. Educational Psychology Review, 8, 357-371.

Mayer, R.E. (1999). Designing instruction for constructivist learning. In C.M. Reigeluth (Ed.). Instructional-design theories and models. Volume II. A new paradigm of instructional theory (pp. 141-159). Mahwah, NJ: Lawrence Erlbaum Associates.

Mayer, R. E. (2001). Multimedia Learning. New York: Cambridge University Press.

Rothkopf, E.Z. (1970) "The concept of mathemagenic activities" Review of Educational Research, 40, 325-336.

Peeck, J. (1993). Increasing picture effects in learning from illustrated text. Learning and instruction, 3, 227- 238.

Rothkopf, E.Z. (1996) Control of mathemagenic activities. In D.H.Jonassen, (Ed.), Handbook of Research on Educational Communications and Technology. New York: McMillan.

Schnotz, W., & Bannert, M. (2003). Construction and interference in learning from multiple representation. Learning and Instruction, 13, 141-156.

Twohig, P.I. (1982). The effects of examples on the comprehension of textbooks. Virginia: University of Virginia, Department of Psychology. (Paper presented at Midwestern Psychological Association, 1982, Minneapolis).

Van Dijk, T.A., & Kintsch, W. (1983). Strategies of Discourse C²omprehension. New York: Academic Press.

Wade, S. (1992). How Interest Affects Learning From Text. In K. A. Renninger, S. Hidi, and A. Krapp (Eds.), The Role of Interest in Learning and Development (pp. 255-278. Hillsdale, NJ: Lawrence Erlbaum Associates.

Winne, P.H. (in press). Students’ calibration of knowledge and learning processes: Implications for designing powerful software learning environments. International Journal of Educational Research.