International
Journal of
Educational
Technology

Alliance Prototype Distributed Learning Environment:
Emerging Technologies for Science, Education, and Business

Alaina G. Kanfer, National Center for Supercomputing Applications
Umesh Thakkar, National Center for Supercomputing Applications
Jonathan A. Moore, National Center for Supercomputing Applications
Josh Michaels, National Center for Supercomputing Applications

The development of advanced distributed learning environments has the capacity to transform educational and social processes for learning and teaching. One prototype of this kind of environment is the "Emerging Technologies for Science, Education, and Business" course developed by the National Center for Supercomputing Applications (NCSA's) Education Division in collaboration with NCSA's Media Group during the spring semester of 1998. The goals of the course were threefold: (1) to provide opportunities for graduate students to learn about emerging Alliance technologies, in a classroom setting, (2) to provide opportunities for students to understand and engage in cross-disciplinary research activities and (3) to prototype technologies for distributed learning environments. Fourteen students from a range of disciplines were enrolled in the course, which met, face-to-face once a week for 3 hours. Each class session was Web cast in real-time. Course materials, including video and slide presentations, readings, and student assignments were archived online. Thus the course described in this paper is a first step in prototyping the delivery of leading edge content via advanced technologies. In this article, we will summarize course evaluation data and our observations of successes and obstacles as well as outline how future versions of the emerging technologies course will incorporate lessons learned from this initial course offering and experimental technology.

Background: What is the National
Computational Science Alliance (The Alliance)?

On October 1, 1997, the National Science Foundation (NSF) Partnership for Advanced Computational Infrastructure (PACI) initiative funded two partnerships: the National Computational Science Alliance (Alliance) and the National Partnership for Advanced Computational Infrastructure (NPACI) for the next five years. This program is, in many ways, a continuation and expansion of the previous supercomputing program funded by NSF during the previous decade. Each partnership is structured with a "leading edge site", and over fifty partnering universities, government sites, and industrial partners working together to develop America's information and computational infrastructure for the 21^st Century. The National Center for Supercomputing Applications (NCSA) is designated the leading edge site for the Alliance (see http://alliance.ncsa.uiuc.edu). The goal of the Alliance is to prototype a National Technology Grid (Grid) that will provide desktop access to distributed advanced computational science and engineering problem-solving environment to its scientists, engineers, students, and industrial, government, and strategic partners (Smarr, 1997).

The Alliance is organized into four major teams: Applications Technologies (AT), Enabling Technologies (ET), Partnerships for Advanced Computational Services (PACS), and Education, Outreach, and Training (EOT). The AT teams, which are organized into six science disciplines (chemical engineering, cosmology, environmental hydrology, molecular biology, nanomaterials, scientific instrumentation) drive the development of the Grid. The ET teams, which are organized into three computer science teams (parallel computing, distributed computing, data and collaborative computing) build new tools. The PACS are composed of major universities and state organizations to educate and train a broad user constituency of the Alliance. The NSF PACI program has created a national EOT program spanning the Alliance and NPACI partnerships (see http://www.eot.org). These national EOT teams work with AT and ET teams to integrate computational science into learning activities at all levels.

Why offer an emerging technologies course?

As the Alliance scientists and engineers continue to develop the Grid to provide desktop access for the 21^st century, considerations about the needs and capabilities of future users of the Grid arise. For instance, how do we prepare today's students for the scientific and technological developments of the 21^st century? What opportunities are available to the applications technologies (AT) and enabling technologies (ET) scientists to participate in education and training of students today? How can university students learn about the leading edge developments at the Alliance? As the Grid development will continue to integrate new technologies over time, there needs to be a mechanism whereby the impact of the Grid technologies on research, education, business, and social processes are examined not only by the developers but also by future users (e.g., today's students). Furthermore, we need to be aware of the social and ethical implications of emerging technologies. However, typically graduate students and upper level undergraduate students learn about Grid technologies only if they are directly involved in an Alliance project as a research assistant.

As a starting point to address at least some of these issues, NCSA's Education Division in collaboration with NCSA's Media Group developed the Emerging Technologies for Science, Education and Business course. The initial goal of the course was simply to provide a systematic introduction to information about emerging advanced computational science and communications technologies and applications to graduate students and the course was conceptualized as a "best-of-the-Alliance" style seminar series. The hope was that this classroom mechanism would provide a structured way for graduate students to learn about emerging technologies as they were being developed, so that they could incorporate such developments and assessments of new technologies into their on-going graduate work, even if they did not hold an appointment with an Alliance researcher.

What were the goals of the emerging technologies course?

Two of the authors, Kanfer and Thakkar were the instructors of record for the Emerging Technologies course. Each held a teaching appointment on campus -- in the College of Commerce & Business Administration and the College of Education, respectively. Thus course was designed for beginning graduate students and upper-level undergraduates who were interested in understanding new technologies and their applications in education and business as well as in science. The course provided an introduction to computational science and to emerging technologies used to support advanced computational science. The course learning objectives were to provide the students with an understanding of the range of technologies and applications being developed at the Alliance and to provoke some thought as to how these emerging technologies might impact the students' individual disciplines. An additional learning objective was to provide students with an understanding of the scale of multidisciplinary teams involved with developing and using emerging technologies.

Through guest lectures, hands-on demonstrations and discussions, the course addressed questions such as, What is computational science? What tools are used for computation? How are computation, simulation, modeling and visualization applied in various fields of science such as cosmology and environmental hydrology? How might digital libraries, virtual reality, and collaborative tools support collaboration? What is the impact of such collaborative tools on learning? What types of professionals are involved in developing new applications of advanced computational resources? How do multidisciplinary teams work together to support the use of emerging technologies for knowledge discovery?

The course was offered for credit to students on the University of Illinois Urbana-Champaign campus. However, it seemed limiting to offer the opportunity to learn about emerging technologies only to UIUC students when we were part of a large multi-university Alliance. Moreover, as we began confirming our list of guest lecturers it became clear that the actual scheduling of such a wide-ranging seminar series was more difficult than we anticipated, and a "best-of-the-Alliance" seminar would be difficult to replicate. Therefore an additional goal of the course was to identify a delivery mechanism whereby students and others at our partner sites could also learn, in real-time, from our on-site classroom, and whereby we could create a useful asynchronous archive of the emerging technologies course.

Thus the overall goals for the course were (1) to provide opportunities for graduate students to learn about emerging Alliance technologies, in a classroom setting, (2) to provide opportunities for students to understand and engage in cross-disciplinary research activities and (3) to prototype technologies for distributed learning environments.

How was the emerging technologies course delivered?

On campus, the course was cross-listed in three colleges: education; commerce & business administration; and agricultural, consumer and environmental sciences. The fourteen students (1 female, 13 male) who completed the course represented a range of disciplines and academic levels. Seven of fourteen students were business students working on their MBA degrees. Three additional masters-level students were from computer science, educational administration and supervision, and writing studies. Three doctoral students were from curriculum and instruction, food science and human nutrition, and leisure studies. One undergraduate registered for the course was majoring in psychology. With such a wide range in academic majors and grade levels, every attempt was made to encourage cross-disciplinary group projects among the students.

Each class session during the fifteen-week semester focussed on an area of research related to enabling technologies or applications technologies teams of the Alliance, with the exception of the last class session, which was devoted to student project presentations. Typical class sessions included two to three guest lecturers, demonstrations and discussions related to the topic. If time permitted, relevant computer-based activities were included during class to increase the students' understanding of the topic (see http://www.ncsa.uiuc.edu/edu/courses/spring98/lecturers/week/ for a list of topics and lecturers for each week). Forty guest lecturers from the Alliance participated in the course (see http://www.ncsa.uiuc.edu/edu/courses/spring98/lecturers/speaker/ for an alphabetical listing of lecturers). Guest lecturers worked with instructors to select readings related to their topics, which were distributed to the students one week in advance. An electronic mailing list was established to inform students about upcoming activities such as additional lab sessions and assignment deadlines.

As an example, a session on distributed learning environments included four guest lecturers and focussed on new initiatives (see http://www.ncsa.uiuc.edu/edu/courses/spring98/lecturers/week/topics/week4.html), such as a discussion on UI-OnLine and a demonstration of netLearningPlace. UI-OnLine is an organization established to support three University of Illinois campuses on their educational and public service offerings over the Internet. netLearningPlaceÔ is an NCSA developed collaborative learning environment. In addition, this session included discussions of cognitive, instructional, organizational, and technical issues in distributed learning and teaching. While the students were exposed to a wide range of content in this class session, there was little time for the students to actually have hands-on experience with various online learning technologies in the classroom. In contrast, the class session on chemical engineering (see http://www.ncsa.uiuc.edu/edu/courses/spring98/lecturers/week/topics/week7.html) included only two guest lecturers and the students had time to explore the ChemViz project online at http://www.ncsa.uiuc.edu/edu/chemviz/ after listening to presentations by the guest lecturers.

There were two assignments the students had to complete during the semester; one individual and one group assignment (see http://www.ncsa.uiuc.edu/edu/course98/assignments). For the individual assignment, each student was responsible for summarizing the content of one class session, and to propose relevant discussion questions and speculate on how that content might be relevant to his or her own field. The group assignment was a semester long project, and groups of any size were allowed. We saw groups from size one to four students. The group project was fairly open-ended. The students were instructed to "develop a project or a research proposal relating to the information presented in the course." Examples were provided to prompt them, such as a review of Web-based collaborative technologies, a report on the current and future uses of technology in your particular discipline or a research proposal to examine how geographic information systems could enhance student learning about ecology in middle school classrooms. The overall goal of the project was to encourage the students to work in interdisciplinary teams and to think creatively about the technologies they were learning about in the class.

The course was offered on-site at NCSA's PC lab. The lab has twenty student stations arranged in three rows facing the front of the room, providing each student with an individual computer running a Pentium pro processor with 17-inch monitors. There is an additional instructor computer at the front of the room equipped with both a wireless mouse and an air mouse. Each of the twenty-one computers in the lab is connected via switched Ethernet to the university backbone providing full Internet access. The machines run Windows NT 4.0 with full suite of applications including Microsoft Office 97, Netscape, Internet Explorer, and Exceed among others. A large projection screen is in front of the classroom, which can project the instructor's computer, videotape, or even one of the students' computers. In addition, each of the computers in the room can drive all of the other monitors so that students can see the instructor computer at their own desktop, and students can share their own views with one another (see http://www.ncsa.uiuc.edu/edu/courses/spring98/about/index-tech.html.)

A course Web site was established for the dual purposes of supporting the on-site course and making the course information available to people outside of the UIUC campus (see http://www.ncsa.uiuc.edu/edu/course98). The Web site includes a section about the course, including the course syllabus and requirements, and technical set up. In addition, each week the Web site was updated to include the agenda for that class session, recommended and required readings and links. The Web site also posted assignments for the course, and the students work was shared online through the site. In addition, each week, the guest lecturer was assisted in preparing and/or uploading slides and additional resources to the course Web site so that the online repository was kept as up-to-date as possible.

When the course was being planned in December 1997, we approached NCSA's media group about the possibilities of Web casting, or streaming video live from the classroom when class began in January 1998. However, the Media group had not experimented with this technology yet, and NCSA did not even own the card needed to encode the video for the Web cast. Plans were set for NCSA to begin Web casting at our annual meeting in April of 1998. However, the day before the first day of class, in January 1998, the encoding card arrived. Moreover, NCSA's director, Larry Smarr was the first guest lecturer in the emerging technologies course, which helped to elevate the status and priority of Web casting the class session. So for the 24 hours before class began extensive testing took place, and on the afternoon of January 22, 1998 NCSA's first Web cast took place, successfully, from the PC lab during the first class meeting of the emerging technologies class.

The Web casting of the first class session set in motion a process of the Media group spending Thursdays moving equipment over from their building to the PC lab, setting up the Web casting, running the Web cast live taking down after class, and archiving the video files of the lectures within two days of class each week. The equipment used to Web cast each week included an HP Kayak Pentium II machine with video encoding card, two BetaMax recorders, lights, and a scan converter. In addition to the cables to hook all this up, each week, the Media Group set up a switcher so that in real-time, a "producer" could select the feed into the Web cast from the various sources (camera, instructor computer, videotape, etc.). The final broadcast was 200 kilobits per second over the Internet using RealNetworks' RealVideo encoder and server. Initially the Web cast was broadcast at approximately five frames per second, and then after installing a second processor into our encoding machine at approximately twelve frames per second. In addition to archiving the entire guest lecture, each guest was asked to provide a brief (30 second) summary of his or her presentation at the end of his or her guest lecture. These summaries were used as video abstracts of the talks, and are archived on the course Web site in addition to lecture notes and the full length video.

How did the students evaluate
the emerging technologies course?

Two types of evaluation data were collected from the 14 students enrolled in the course during the semester. For each session, students provided feedback through guest speaker evaluation forms that requested students to identify the best parts of each guest lecture, and provide suggestions for how to improve each guest lecture. The weekly guest speaker evaluations were recorded on paper forms. Students used this opportunity to evaluate the topics included in the course, as well as various speakers' presentation styles. Responses ranged from "interesting use of media. Was able to put things into perspective, both across disciplines and over time" and "extremely interesting and easy to follow and great slides - wish he had more time" to "this topic was of no value to me. Did not clearly convey, in a matter that could be easily understood, what he was talking about."

In addition to the guest speaker evaluation forms, an online evaluation form was developed for the overall course evaluation (see http://www.ncsa.uiuc.edu/edu/course98/eval). Twenty-eight of the total thirty questions were incorporated from the standard course evaluation forms used at the University of Illinois at Urbana-Champaign (Instructor and Course Evaluation Survey, ICES, developed by UIUC Office of Instructional Resources). Each of these questions elicited responses on a five-point rating scale to a statement regarding the course content, structure, materials, workload and overall attitudes toward the course. Two additional open-ended questions were added for students to identify three individual presentations most beneficial to them and three class sessions most relevant to them. Eleven of fourteen enrolled students completed the online course evaluation. The mean and standard deviation of responses to these course evaluation items (treating the ordinal rating scales as an interval scales) are included with the questions in the appendix of this paper.

Through the course evaluation responses, we hoped to gain some insight into how well we achieved our learning objectives of developing (1) an understanding of the range of technologies and applications being developed at the Alliance and consideration of how these emerging technologies might impact the students' individual disciplines and (2) an understanding of the scale of multidisciplinary teams involved with developing and using emerging technologies. Note we did not use this evaluation method to assess the degree to which we accomplished our third goal of prototyping an environment for online learning.

Among all the questions, the most extreme responses were to items 1 and 28, "Did the course duplicate other courses in this area" (mean = 4.6 anchored with 5 = not at all), and "I learned more from the readings than I did from lectures and class discussion (mean = 4.3 anchored with 5 = strongly disagree). These two questions indicated that the course content was new to the students and perhaps the structure of the course was not balanced. Further investigation of the responses suggests that the students felt that they learned a lot (mean = 1.6, anchored with 1 = A great deal) and that the objectives of the course were accomplished (mean = 1.64 anchored with 1 = Yes, to a great extent). The pace of the course seemed about right (mean = 2.8 anchored with 1 = too fast) and the workload was reasonable. However, the multidisciplinarity of the course might have made it a little too broad (mean = 2 anchored with 1 = too broad). Moreover, while the students were clear that the content of the course was relatively new, their responses to how the new knowledge might be put to use were not quite as favorable. For instance ratings averaged 2.0 to the question "Did you improve your ability to evaluate new works in this field", anchored with 1 = Yes, greatly, and Responses to "This course gave me the opportunity to develop some original ideas" averaged 2.18, anchored with 1 = To a great extent.

Thus it seems that our goal of introducing students to a range of emerging technologies was achieved. On the other hand, the course evaluation responses, while definitely positive, suggest that we did not accomplish our goal of providing multidisciplinary learning opportunities as effectively as we introduced the range of technologies. On the positive side overall, there was a good consensus that the students would recommend the class to other students. The mean degree to which students would recommend the course was 1.6 with a standard deviation of 0.6. Finally, the three class sessions that were most relevant to the students were visual computing and virtual reality, distributed learning environments, tools for data and collaborative computing.

What successes did we observe in
the emerging technologies course?

Our first success in offering the emerging technologies course was that we actually had more than fourteen students registered in the course which was announced only two weeks before the first class session. Second, we were fortunate to have such cooperative colleagues that we were able to assemble forty dynamic and enthusiastic guest lecturers to participate in the class over the semester. Much of our effort was devoted toward developing an efficient protocol for obtaining appropriate speaker information, lecture notes, photographs, readings, video on the website and installing any requisite software in the classroom.

With respect to our goals of providing multidisciplinary learning experiences, while we did not have a very even gender distribution, the students in the course represented a fairly good range of majors and grade levels. Moreover, the group projects were fairly multidisciplinary. Students formed their own groups. With about one half of the students in class from the MBA program, they ended up forming two groups comprised completely of MBA students. Those groups studied collaborative tools and electronic commerce respectively. On the other hand, the rest of the student group projects exemplified the benefits of multidisciplinary research. For instance, the psychology undergraduate student paired with the English graduate student on an "Interface Design and Analysis for the Collaborative Writing Engine (CWE)." This project provided the psychology student with real-life experience and provided the English student with much needed input from someone trained in usability issues. Another successful collaboration among students in the class involved a Ph.D. student in music education and a Ph.D. student in leisure studies. For their class project they submitted a proposal, "The Effect of Web-Based Streaming Media on Destination Image and Knowledge." Although they had never worked on a project together, this proposal read as if the two students had already been collaborating. The final multidisciplinary project, "Research Proposal for Comparative Evaluation of Alternative Instructional Delivery Methods," was submitted by a Masters student in computer science and a Masters student from the College of Education. The one-person group consisted of a Ph.D. student in food science whose interest is in food service employee training via video and Internet.

The third success we observed was the prototype development of a distributed learning environment. This involved both the Web site we developed and the weekly live Web cast and video archive. During the first class session, over 100 Internet Protocol addresses logged into the video server watching the class live - and we only announced the streaming test with the URL that day and only to our partners! Throughout the semester, the server logs revealed continued access to the video archive. We estimate that over 800 video files were downloaded throughout the semester, with almost two-thirds of the requests coming from outside of NCSA. In addition, over 300 video files were requested in the month after class. Again, only one-third of the requests for video files came from NCSA host computers. Thus we the Web site was serving as a prototype distributed learning environment, and we were reaching sites outside of NCSA.

What obstacles did we observe in
the emerging technologies course?

There were several obstacles we encountered in offering the first emerging technologies course. One set of obstacles seemed to be related to trying to include too much information in one semester. In terms of the content, it almost seemed that we were trying to offer two courses in one semester; one course on technologies and a second course on science applications of the technologies, with a little research and evaluation sprinkled throughout. For some topics, such as visualization, there was such a high level of interest that we easily could have spent two or more weeks on that topic alone. The students also did not have very strong backgrounds in the physical or biological sciences and so during the weeks focussing on science applications of technologies, the students would have benefited from an introductory session providing an overview of the science area. Informal discussion with students confirmed this observation, as did the response pattern on evaluation forms. Moreover, the lecture content filled most of the class sessions so that there was not enough time left for hands-on activities. In addition, we noted a particular student sensitivity to lecture style in computer classroom especially when the camera was running in the room. Students responded more positively to lecturers who demonstrated a comfort with the technologies in the wired classroom, and used them during their presentation.

There were technical difficulties associated with having a large number of guest lecturers. Guest lecturers were not always familiar with the lab. Occasionally, a guest lecturer planned to use special software in the lab for a demonstration but it was not compatible with the lab set-up and so the software would not work. These problems were in addition to the standard equipment problems that we typically encounter in a classroom that relies upon so much technology. The Web casting presented several obstacles; it used a lot of bandwidth and sometimes slowed other applications and the building network. Web casting usually required at least four technical support staff on hand during each class session. The equipment for webcasting was not permanently part of the lab so set up and take down time took an additional four hours. In addition, questions asked by local students could not be heard over the webcast or in the archived video, unless the lecturer remembered to repeat the question for the camera. Finally, there was no mechanism in place for remote students or others watching the lecturers on the web to ask questions or be involved in the class.

What are our plans for future emerging technologies
courses and distributed learning environments?

In spring 1998 course offering of emerging technologies we had three primary goals of (1) trying to deliver innovative Alliance content to graduate students in a classroom setting, (2) trying to create a environment for multidisciplinary learning and (3) prototyping a distributed learning environment. Based on what we learned from our experiences and from the student evaluations we planned three primary changes in future emerging technologies course offerings: (1) content-based (2) activity-based or course structure (3) enrolling online students. Student evaluations as well as our personal experiences suggested that the course material could have been spread over at least two semesters instead of in a single semester. Therefore the first change we implemented in response to the evaluation of the spring 1998 course was to decrease the amount of content covered in the course. The first semester emerging technologies course in the 1998-1999 school year was focussed upon Alliance leading edge enabling technologies such as visualization, virtual reality and collaborative tools and the development, use and impact of these technologies. The second semester was focussed upon the scientific applications technologies, specifically how the scientific modeling and visualization development for the applications teams of the Alliance could be applied as educational prototypes for learning and teaching in K-12 and university classrooms (see http://www.ncsa.uiuc.edu/edu/courses for later course offerings).

Course evaluations also revealed that too much time was spent on guest lectures, leaving little time for students to actually have hands-on experiences in class with the technologies being introduced. Therefore, in addition to decreasing the amount of content covered in the second offering of emerging technologies in fall, 1998, the second change we implemented was to reduce the number of guest lecturers. This left more time to plan in-class discussions, and hands-on activities with emerging technologies.

The third change we implemented was to involve online participants in our on-site classroom. Recall that we observed people from outside NCSA requesting videos of the guest lectures throughout the semester. These outside visitors had no way to ask a question or otherwise interact with the students or instructors in the classroom. Moreover, the content of the emerging technologies course is relatively unique and not offered at other universities. Therefore, in addition to offering the emerging technologies courses to UIUC students we planned to offer the course online to students who are enrolled at other universities. The concept of offering a distributed learning course, or even series of courses online in a program, is not new. For example, the UI-OnLine initiative at the University of Illinois was established to provide an opportunity to remotely enroll in regular university courses for students in Illinois who for various reasons cannot attend class on campus (Manning, et al., 1998). The LEEP program in the University of Illinois Graduate School of Library and Information Science is a completely online masters program with students enrolled from all over the United States (http://alexia.lis.uiuc.edu/gslis/leep3/). The New Jersey Institute of Technology has been offering distributed learning courses via technology for over ten years. The remote students participate asynchronously and the NJIT offers degrees in Information Systems and Computer Science degrees online (Hiltz, 1997). The difference in our plans, we that we were still holding a traditional on-site class, and that we planned to integrate online students directly into the on-site classroom, synchronously and asynchronously. This was administratively possible by offering the emerging technologies course as a pilot class in Common Market of Courses and Institutes (CMCI) program (http://www.cic.uiuc.edu/CMCI/cmci_homepage.htm) of the Committee on Institutional Cooperation (CIC). The CIC consists of the eleven members of the "Big Ten" and University of Chicago.

At NCSA we also get many requests for workshops from working professionals in industry and government on Alliance technologies. Research on distributed learning environments suggests that distributed learning environments may be especially practical for working professionals seeking continuing education and lifelong (Hiltz, 1997; Mayadas, 1997). Furthermore, in the business world a current trend sees a significant reduction in the amount of time available for training employees. Instead, businesses are depending more and more on "just-in-time training." Just-in-time training refers to training employees what they need when they need it. This trend reduces the tendency to encourage employees to enroll in complete academic courses. Rather, employees are given mini-lessons or smaller sections of a larger topic, limiting work down-time, as the employee is only absent for the time necessary to learn what is needed to be learned (Gordon, 1997; Mayadas, 1997). Expanding on this concept of the mini-lesson or just-in-time training for the business professional, we organized our course content into three modules, each corresponding to approximately one-third of the 15 week semester and each with its own domain within the field of emerging technologies. Enrollment into these modules was offered to a limited number of NCSA's partners. Therefore, subsequent emerging technologies course offerings were available to two types of online students: graduate students at other universities and working professionals taking the course from their desktop.

Providing more hands-on activities in the course, and integrating online participants into the on-site classroom required additional changes in the technical infrastructure for the course. While we planned to continue Web casting classes each week, it was not sustainable to have the Media group bring in the Web casting equipment each week from another building, come rain or shine. Therefore we purchased the necessary equipment so that Web casting capabilities are permanent in our PC lab. Furthermore, we expanded the technical infrastructure for course to include online videoconferencing and collaborative tools with a range of software applications to facilitate both synchronous and asynchronous learning, all within a password protected Web-based learning environment, netLearningPlaceÔ , developed at NCSA.

Finally, scaling up a prototype project is not always easy. Therefore, we developed a research agenda for understanding distributed learning and teaching in future offerings of the emerging technologies course (e.g., Dede, 1996). In future course offerings we have attempted to encourage synergistic group projects between remote and on-site students to foster collaborative learning (e.g., Haythornthwaite, 1998; Hiltz, 1997; Robinson, 1992). Subsequent versions of our course design have allowed us to investigate the integration of online students into the on-site classroom and the integration of lifelong learners into the graduate classroom with audio and video synchronously. In addition, we have explored the effectiveness of using previous course lecture videos as "readings". Our goal is for the course to continue evolving as an Alliance prototype for distributed learning, including both advanced synchronous and asynchronous environments. Results from this later research will be reported in future articles.

References

Dede, C. (1996). Emerging technologies and distributed learning. The American Journal of Distance Education, 10(2), 4-36.

Gordon, J. (1997). Infornuggets: The bite-sized future of corporate training? Training, 34(7), 26-33.

Haythornthwaite, C. (1998). A social network study of the growth of community among distance learners. Internet Research, 4(1). [Available online: http://www.shef.ac.uk/~is/publications/infres/infres41.html]

Hiltz, S. R. (1997). Impacts of college-level courses via asynchronous learning networks: Some preliminary results. Journal of Asynchronous Learning Networks, 2(1). [see also http://www.aln.org/alnweb/journal/jaln_Vol1issue2.htm]

Hiltz, S. R., & Wellman, B. (1997). Asynchronous learning networks as a virtual classroom. Communications of the ACM, 40(9), 44-49.

Manning, S., Oakley, B. & Ward, L. (1998) "UI-OnLine: The Realization of the 21st Century Land-Grant University." [Available Online at http://www.online.uillinois.edu/about.uionline.html]

Mayadas, F. A. (1997). Online networks build time savings into employee education. HRMagizine, 42(10) 31-35.

Robinson, R. (1992). Andragogy Applied to the Open College Learner. Research in Distance Education, 4(1), 10-12.

Smarr, L. (1997). Toward the 21^st century. Communications of the ACM, 40(11), 29-32.

Each of the following five-point evaluation items were adapted from the UIUC Instructor and Course Evaluation System. The online version of the evaluation form is available at http://www.ncsa.uiuc.edu/edu/course98/eval and the questions are reproduced here together with the scale anchors, means and standard deviations. There were eleven respondents for each question.

Final Evaluation (May, 1998)

Please fill out the following form to provide us with some input about the NCSA Course on Emerging Technologies. We will use this information to help us plan future courses. Thank you. If you have a question about filling out the survey, or encounter a problem while submitting it, please contact either Umesh Thakkar, Alaina Kanfer or Josh Michaels for assistance.

1. Did this course duplicate other courses in this area?
   Considerably - Not at all
   4.6 (0.70)
   
2. How well did the instructors coordinate different activities of this course?
   Very well -Rather poorly
   2.18 (0.87)
   
3. This course was creatively planned.
   Strongly agree - Strongly disagree
   1.64 (0.54)
   
4. I was disappointed with the topics emphasized in this course.
   Strongly disagree - Strongly agree
   2.36 (1.03)
   
5. Was class time spent on unimportant and irrelevant material?
   Yes, often - No, never
   3.0 (1.18)
   
6. The course struck a good balance among reading, discussion and writing.
   To a great extent - Not at all
   2.91 (1.04)
   
7. The scope of this course was:
   Too broad - Too narrow
   2 (0.77)
   
8. The course content was:
   Too advanced - Too elementary
   3.18 (0.60)
   
9. The course content was:
   Too theoretical - Too applied
   3.1 (0.57)
   
10. The readings were:
    Extremely difficult - Extremely easy
    3.11 (0.60)
    
11. Were reading assignments relevant to class presentations?
    Yes, always - No, almost never
    2.2 (1.40)
    
12. Did supplementary text(s) help you expand your knowledge of the material?
    To a great extent - Not at all
    2.2 (0.92)
    
13. Describe your written assignments.
    Interesting, stimulating - Dull, uninspiring
    2.5 (0.97)
    
14. Completing written assignments was a good use of my time and effort.
    Strongly agree - Strongly disagree
    2.5 (1.08)
    
15. How much work did this course require?
    Excessive amount - Not enough
    3.4 (0.70)
    
16. The amount of outside preparation required for this course was:
    Quite great - Quite small
    3.45 (0.69)
    
17. Was the course appropriate for your background (experience)?
    Very appropriate - Not at all appropriate
    1.6 (0.97)
    
18. Describe the pace of the course.
    Too fast - Too slow
    2.8 (0.63)
    
19. How much do you feel you have accomplished in this course?
    A great deal - Very little
    2.3 (1.16)
    
20. Compared to other courses, how much did you learn in this course?
    Much more - Much less
    1.90 (0.74)
    
21. How much have you learned in this course?
    A great deal - Very little
    1.6 (0.52)
    
22. Do you feel course objectives were accomplished?
    Yes, to a great extent - No, not at all
    1.64 (0.50)
    
23. Would you recommend this course to other students?
    Highly recommend - Not recommend
    1.64 (0.67)
    
24. Did you improve your ability to evaluate new works in this field?
    Yes, greatly - No, not really
    2.0 (0.63)
    
25. This course gave me the opportunity to develop some original ideas.
    To a great extent - Not at all
    2.18 (.87)
    
26. The course gave me skills and techniques directly applicable to my career.
    Strongly agree - Strongly disagree
    2.2 (1.23)
    
27. Did you read required books or other materials suggested in the course?
    Strongly agree - No, none
    3.0 (0.89)
    
28. I learned more from the readings than I did from lectures and class discussions.
    Yes, all - Strongly disagree
    4.3 (0.82)
    
29. Please list the three individual presentations that benefitted you the most.
    Click here for an index of speakers from Week 1 through Week 12.
    
    Please note: Week 13 and 14 speakers are not currently indexed. These speakers were Roy Campbell (Week 13), NCSA Course Tech Staff (Week 13), Chip Bruce (Week 14), Jim Levin (Week 14) and Nick Burbules (Week 14.)
    
    [Incomplete data - results not reported]
    
30. Please select three class sessions that were most relevant to you.

IJET Homepage| Article Submissions| Editors | Issues

Copyright © 1999. All rights reserved.
Last Updated on 1 December 1999. Archived 5 May 2007.
For additional information, contact IJET@lists.ed.uiuc.edu