Online, blended, and classroom teaching of economics principles: A randomized experiment

This paper contains estimates based on the first three semesters of a field study of learning
outcomes and delivery modality for Principles of Economics. The experimental design randomly
assigns students to one of three delivery modalities: online, blended, or classroom-based
instruction. In the classroom, students meet with an instructor, once in a lecture session and once
in a discussion session. In the blended format, students meet weekly with the instructor in a
discussion session, and view online lecture materials. In the online course, students view online
lecture materials as their course instruction and have access to extensive online materials
developed to be consistent with a set of external standards for best practices in online education.
For the three arms of the experiment, lectures, discussions, and other instructional content are
prepared and delivered by the same instructor.

With University Institutional Review Board approval, data is being collected on academic achievement, demographic characteristics, and personality traits on four consecutive semesters. The class size for each arm of the experiment is capped at 35. The experiment began in fall 2012. The three semesters of data available for this paper consist of observations on approximately 350 students. Approximately 250 students completed the course, and another 100 either dropped the course or chose to withdraw from the experiment before the course began. The fourth and final semester of data collection is being collected during the spring 2014 term. Power analyses were conducted prior to running the experiment to assure we would be able to detect reasonably small changes in academic outcomes across the class types of the experiment. 

Measures of learning outcomes are graded cumulative final exams and the Test of
Understanding College Level Economics (TUCE). Preliminary results based on three semesters
of data suggest the randomization strategy has largely been successful. The learning outcomes on
exam scores are not, at a statistically significant level, different between the control group
(traditional modality) and blended treatment group. However, the exam scores for the control
group are higher, at a statistically significant level, than for the online treatment group. The
preliminary results suggest that if equivalence of learning outcomes is a goal, then as higher
education moves forward with online education, more instructional resources should be directed
at online relative to traditional instruction (e.g., smaller section size for online).