Abhijeet Singh, Stockholm School of Economics. "Auditing and Improving Education Data in Developing Countries"
Department seminar. Abhijeet Singh is an Assistant Professor at the Stockholm School of Economics. He will present his paper: The Myths of Official Measurement: "Auditing and Improving Education Data in Developing Countries"
Large-scale student assessments are central to global education policy goals but vulnerable to cheating by students and teachers. In this paper, I study the severity of cheating-induced distortion in census-based assessments in India, using unique audit studies, and evaluate methods to reduce it at scale. I first focus on an annual standardized test in government schools in Madhya Pradesh state, which has covered nearly 7 million students annually since 2011, and is seen as a national “best practice”. Comparing question-specific responses of students, as reported in the official data, to an independently-conducted retest audit with the same test items, I show that student achievement levels are severely inflated in official data. This distortion affects students at all levels of ability, is especially pronounced for weaker students, and is at least partly teacher-induced: distortion is about 40% lower in grades where student exam scripts were, by design, sent to other schools for grading and the test responses also have greater predictive power. In a follow-up study covering over 2400 schools in a different state (Andhra Pradesh), I evaluate whether tablet-based testing, which makes cheating harder for both students and teachers, could reduce distortion. As in MP, official test data from paper-based assessments, in both private and government schools, severely exaggerate student achievement in comparison to a retest audit. In contrast, there is no evidence of such distortion in tablet-based assessments. These results suggest that, in settings of weak governance, business-as-usual learning assessments are likely to be compromised even without high-powered incentives; however, computer-based testing may provide a scalable solution which can be implemented within government systems.