86th International Atlantic Economic Conference

October 11 - 14, 2018 | New York, USA

Can we stay one step ahead of cheaters? A clinical trial to find the effect of proctoring in an online exam

Friday, 12 October 2018: 2:40 PM
Jose Vazquez, Ph.D. , Economics, University of Illinois–Urbana-Champaign, Urbana, IL
Ignacio Sarmiento Barbieri, Ph.D. Candidate , Economics, University of Illinois at Urbana-Champaign, Urbana, IL
Online assessment strategies offer instructors several key advantages over comparable analog (pencil and paper), ones. Yet, perhaps the most difficult barrier preventing instructors from adopting these technologies has to do with their effect on academic dishonesty. In other words, instructors are leery of using them because they feel it is an invitation for students to “cheat” in the assessments. Yet, this assertion has had little rigorous testing in the literature. This paper begins to address that by using a field experiment to test the effect of proctoring on an online assessment.

We conducted a randomized controlled trial (RCT) to estimate these effects over two courses of Microeconomic Principles at a large US public university. The first experiment was conducted over 16 weeks in a traditional face to face Principles of Microeconomics course in the Spring of 2017. The second experiment was over a 4-week online Principles of Microeconomics course in the Winter of 2017. In total 495 undergraduate students participated of the experiment. To identify the average causal effects of taking the exam with a proctor we estimate by OLS a model where the dependent variable is the grade that a student obtained in said exam. The main independent variable is an indicator variable that takes one if the student took the exam with a proctor. This allows us to compare the average differences in grades between students that took the exam with a proctor from those that didn’t. Although we have a randomized set up, to address the potential concern that the student performance on the test may be correlated with the presence of a proctor (e.g. the presence of a proctor may generate anxiety on the test taker) we include a vector of student level controls that include demography characteristics, their class, and ability proxies.

Our results suggest that while proctoring does have a statistically significant effect on reducing cheating, this effect is quite different depending on the level of the student’s ability. Furthermore, we were able to differentiate types of cheating, such as ghost exam takers, as well as students passing information from the answer key with each other. We hence conclude most of the cheating took place in the form of students from the same course collaborating with each other. These results have implications for any instructor considering using online exams in their courses.