ASCILITE 2004: Cheung - fun and games with mobile phones: SMS messaging in microeconomics experiments

[ ASCILITE ] [ 2004 Proceedings Contents ]

Fun and games with mobile phones: SMS messaging in microeconomics experiments

Stephen L. Cheung
School of Economics and Political Science
The University of Sydney

This paper details the application of SMS messaging to facilitate 'classroom experiments' in microeconomics classes. Laboratory style experiments and simulations have long been used for teaching and learning in many disciplines, however classroom experiments in economics have a feature that distinguishes them from their counterparts in many other disciplines. This is that students participate in dual roles as both experimental subjects and as researchers in analysing the outcomes of an experiment. Good experimental designs should thus support student learning with feedback at both levels. In the past, this was done most effectively by conducting experiments in fully networked laboratories. However in view of the cost of developing fully computerised experiments, paper based designs have remained prevalent, even though these are cumbersome and time consuming to conduct. By using SMS as a response medium, the limitations of paper based experiments are overcome without incurring the costs of full computerisation. Individual feedback is delivered by return messaging, while aggregate results are summarised in automated charts and tables. This makes it possible to conduct classroom experiments in large classes in lecture halls.

Classroom experiments in economics

Classroom experiments are a form of simulation gaming used to demonstrate the properties of markets and models in economics. In an experiment, students act firstly as individual decision makers in an economic interaction, experiencing directly the forces at work behind the theories studied in class. Subsequently, they analyse the resulting patterns of behaviour at a broader level and evaluate outcomes from the perspective of society as a whole. In contrast to simulation activities in other social science fields, classroom experiments in economics have the feature that they are explicitly derived from formal models, such that their outcomes can be compared to the predictions of an underlying theory. Thus key steps in the design and implementation of an experiment are as follows (Noussair and Walker, 1998):

The instructor takes an abstract theoretical model and makes it operational by specifying roles for each participant, a set of decision options, and rules for mapping choices into outcomes or payoffs for each participant. In most interesting applications, the outcome to each participant depends both on their own decision, and also those of one or more of their peers. For example:
- Bargaining game: two players must negotiate the division of a sum of money in a once off interaction. The role of the first player is to propose a division of the money. The role of the second player is to specify the minimum share they are willing to accept (Harrison and McCabe, 1996; Dickenson, 2002).
- Contributions game: each member of a group has a sum of money to allocate between a private account and a group account. Each dollar allocated to the private account yields a specified return to the contributor alone. Each dollar allocated to the group account yields a larger return, but this is shared equally between all members of the group (Leuthold 1993).
Student responses are collected and results are compiled, both for the purpose of reporting feedback to individual students, and to produce aggregate data for use in discussion. While it is desirable for data to be recorded in an electronic format to speed production of feedback, paper based recording remains prevalent in the literature.
In follow up discussion, students are prompted to discuss their reasons for making the choices they adopted. The predicted outcomes of the experiment according to economic theory are revealed, and compared to the data generated in the experiment.

Providing feedback to students

Classroom experiments provide a bridge between the subject matter of microeconomics and the real world. On one hand, students participate in learning activities that mirror their experiences in actual markets. On the other hand, they learn to interpret these experiences within the framework of microeconomics, for example by determining whether a theorised equilibrium is reached or whether the resulting allocation is efficient. This leads to the critical insight (Bergstrom and Miller, 1997; Cheung, 2003) that students participate in classroom experiments at two distinct levels: both as research subjects participating in an experiment, and subsequently as researchers in analysing the results of the experiment. As such, well designed experiments should provide feedback to students at both levels:

As research subjects, students receive individual feedback on their performance in the experiment, for example on the outcomes of their own individual decision making.
As researchers analysing the results of an experiment, students later study aggregate data for the entire experiment, summarising overall patterns of decision making and outcomes to all participants.

In some applications students can record individual level feedback for themselves. However this is not the case for the two examples cited above, in which interaction between participants should occur anonymously. In such cases it is necessary to collect responses from individual students, match them against one another, and report the outcomes confidentially to each student.

In the past, the only way to quickly and efficiently generate both forms of feedback simultaneously has been to conduct classroom experiments in a fully networked laboratory environment. Each student is seated at a separate computer terminal and interacts with a software program which is responsible for collating responses and providing feedback. Computerisation facilitates the control of information and speeds up communication, making experiments simpler and less time consuming to conduct. However as such experiments are costly to develop, their use has largely been confined to instructors who conduct similar experiments for research purposes. Moreover, participation is limited by the size and availability of computer laboratories, and the experiments cannot be conducted in classes held in lecture theatres.

As such, the most prevalent means of conducting classroom experiments remains to do so by hand using pencil and paper. Students record their decisions on printed 'record sheets' which are handed in and analysed manually. This is both time consuming and inefficient, such that aggregate level results are often not compiled until after class, for reporting at a later class meeting. Even individual level feedback can be cumbersome for groups in excess of about fifty students - for example, in the paper based version of the bargaining game (Dickenson, 2002), the instructor must not only physically transfer messages between proponents and respondents, but also keep track of how these are matched to one another. Once again, such experiments are unsuited to large lecture classes. In addition, most standard paper based experiments require a minimum of one hour of class time.

In short, while there is a compelling teaching and learning rationale for classroom experiments in economics classes, their adoption has been limited by what is essentially a technical bottleneck in data assembly and analysis. The challenge is firstly to assemble student responses in an electronic format amenable to efficient processing, and secondly to report feedback on results at both individual and aggregate levels. Infrared based audience response technologies are suited to collecting responses from large numbers of students and generating live displays of results in class, however they do not permit feedback to be sent back to students on an individual basis. However mobile phone messaging technology holds the potential to facilitate fully interactive classroom experiments, even in large lecture classes.

SMS mediated classroom experiments

Figure 1 illustrates how SMS messaging can be used to overcome the limitations of pencil and paper experiments without incurring the infrastructure costs of fully computerised experiments. Students submit responses to a designated phone number, which connects to a gateway from the mobile phone network to the Internet. The instructor downloads incoming messages over the Internet to a personal computer, where a spreadsheet macro is run to filter out invalid responses and simulate interactions between students based on their responses. Return text messages are generated for each student to inform them of their outcome in the experiment and these are uploaded back to the gateway for broadcast to students' phones. For example, in the SMS implementation of the bargaining game, if the amount offered by the proponent is greater than or equal to the minimum amount the respondent was willing to accept, a return text message is automatically generated to inform each that an agreement was reached. Conversely, if no agreement is reached, return messages to that effect are generated instead. This would otherwise be a highly complex and time consuming task if done manually in a large class setting. Finally, the spreadsheet macro also generates tables of summary statistics and graphical presentations of results which can be displayed to students on a data projector at an appropriate point in discussion.

Thus, unlike traditional paper based experiments, no manual intervention is required by the instructor - either in reporting feedback to individual students or presenting results for display - as this is all done automatically from downloaded student data at a single mouse click. This means that SMS mediated experiments can be conducted with only a marginal expense in terms of time, being the time taken to set out instructions at the start of class, and then the time required to download the responses and display results at an appropriate point in class. Since individual students' handsets are used for data entry, no specialised hardware investment is required and the experiments can be run in any classroom from which the instructor can access the Internet.

Figure 1: SMS mediated classroom experiment

During semester one of 2004, four simple SMS experiments - including the two examples cited above - were developed by the author for use in lectures in a core microeconomics unit for postgraduate students of commerce and international business. In each experiment, all analysis, manipulation and reporting of responses is performed by a series of macros within Microsoft Excel, which was chosen because of its inbuilt programmability, charting capabilities, and suitability to processing data organised as lists. In each experiment, students' responses are read in as a comma delimited (CSV) text file containing the originating mobile phone numbers and messages, and an output file is written in the same format containing outgoing messages for upload and broadcast back to students' phones. Provided arrangements can be made for text messages to be uploaded and downloaded in these simple, standard file formats, the experiments can run without modification, thus they are not in principle dependent on any specific service provider. While the ability to download incoming messages in CSV format is a standard feature of most SMS gateway providers, the ability to upload a similar list of replies for broadcast was not available from any known Australian provider. Custom software development was thus commissioned to implement this feature as an add in to an existing application.

The work completed to date has established the technical feasibility of SMS messaging as a medium to facilitate classroom experiments in economics, both in assembling responses from students and for reporting feedback to individual students. Moreover these experiments are suitable for use in large lecture classes, an environment in which options have previously been very limited. Further work is underway focusing on issues of student accessibility, including the possible use of premium SMS services to offset costs to students, and to evaluate student satisfaction and learning outcomes.

Conclusion

Classroom experiments represent one of the most significant and promising developments in the pedagogy of economics over the past decade. However their adoption has been limited by the logistical challenge of collecting and analysing data in large classes without individually networked computers. The work described in this paper demonstrates how a combination of text messaging and spreadsheet processing can be used to overcome the limitations of pencil and paper experiments without incurring the overhead costs of fully networked experiments. In particular, the use of SMS streamlines the process of data entry and assembly, enabling the reporting of results and individual feedback to be automated by a spreadsheet. Moreover, as SMS mediated experiments are designed to economise on both class time and effort to the instructor, they can be readily integrated into large lecture classes.

References

Bergstrom, T.C. & Miller, J.H. (1997). Experiments with economic principles. New York: McGraw-Hill.

Cheung, S.L. (2003). On the use of classroom experiments in 'aligned' teaching. Economic Analysis and Policy, 33(1), 61-72.

Dickenson, D.L. (2002). A bargaining experiment to motivate discussion on fairness. Journal of Economic Education, 33(2), 136-151.

Harrison, G.W. & McCabe, K.A. (1996). Expectations and fairness in a simple bargaining experiment. International Journal of Game Theory, 25(3), 303-327.

Leuthold, J.H. (1993). A free rider experiment for the large class. Journal of Economic Education, 24(4), 353-363.

Noussair, C.N. & Walker, J.M. (1998). Student decision making as active learning: Experimental economics in the classroom. In W.E. Becker & M.W. Watts (Eds.), Teaching economics to undergraduates: Alternatives to chalk and talk. (pp. 49-77). Cheltenham: Edward Elgar.

Williams, A.W. & Walker, J.M. (1993). Computerized laboratory exercises for microeconomics education: Three applications motivated by experimental economics. Journal of Economic Education, 24(4), 291-315.

Author: Stephen L. Cheung (S.Cheung@econ.usyd.edu.au), School of Economics and Political Science, The University of Sydney, NSW 2006, Australia

The author thanks the School of Economics and Political Science for funding the project reported in this paper, Mark Melatos for allowing his class to be used for one of the experiments, and Andrew Grill of LegionONE for facilitating technical development of the software interface.

Please cite as: Cheung, S.L. (2004). Fun and games with mobile phones: SMS messaging in microeconomics experiments. In R. Atkinson, C. McBeath, D. Jonas-Dwyer & R. Phillips (Eds), Beyond the comfort zone: Proceedings of the 21st ASCILITE Conference (pp. 180-183). Perth, 5-8 December. http://www.ascilite.org.au/conferences/perth04/procs/cheung.html

© 2004 Stephen L. Cheung
The author assigns to ASCILITE and educational non-profit institutions a non-exclusive licence to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. The author also grants a non-exclusive licence to ASCILITE to publish this document on the ASCILITE web site (including any mirror or archival sites that may be developed) and in printed form within the ASCILITE 2004 Conference Proceedings. Any other usage is prohibited without the express permission of the author.

[ ASCILITE ] [ 2004 Proceedings Contents ]
This URL: http://www.ascilite.org.au/conferences/perth04/procs/cheung.html
HTML created 22 Nov 2004. Last revision: 22 Nov 2004.