There is growing interest in 鈥榚mbedded experiments鈥, conducted by researchers and policymakers as a team. Aside from their potential scale, the main attraction of these experiments is that they seem to facilitate speedy translation of research into policy. Discussing a case study from Bihar, Jean Dr猫ze argues that this approach carries a danger of distorting both policy and research.
Evidence-based policy is the rage, to the extent that even village folk in Jharkhand (where I live) sometimes hold forth about the importance of 鈥ebhidens鈥, as they call it. No one, of course, would deny the value of bringing evidence to bear on public policy, as long as evidence is understood in a broad sense and does not become the sole arbiter of decision-making. However, sometimes evidence-based policy gets reduced to an odd method that consists of using randomised controlled trials (RCTs) to find out 鈥榳hat works鈥, and then 鈥榮cale up鈥 whatever works. That makes short shrift of the long bridge that separates evidence from policy. Sound policy requires not only evidence 鈥 broadly understood 鈥 but also a good understanding of the issues, considered value judgements, and inclusive deliberation (, 2020a).
Enormous energy has been spent on the quest for rigorous evidence, much less on the integrity of the process that leads from evidence to policy. As illustrated in an earlier contribution to Ideas for India (), it is not uncommon for the scientific findings of an RCT to be embellished in the process. This follow-up post presents another case study that may help to convey the problem. It also illustrates a related danger 鈥 casual jumps from evidence to policy advice. The risk of a short-circuit is particularly serious in 鈥榚mbedded experiments鈥, where the research team works 鈥榝rom within鈥 a partner government in direct collaboration with policymakers.
The case study pertains to an experiment conducted in Bihar in 2012-2013 and reported in Banerjee, Duflo, Imbert, Mathew and Pande (2020)1. This is a large-scale, influential experiment by some of the leading lights of the RCT movement 鈥 indeed, a formidable quartet of first-rate economists reinforced by one of India鈥檚 brightest civil servants, Santhosh Mathew. The high technical standards of the study are not in doubt, and nor is the integrity of the authors. And yet, I would argue that something is amiss in their accounts of the findings and policy implications of this study.
Read More »