Review of Behavioral Economics > Vol 1 > Issue 1–2

Balanced versus Randomized Field Experiments in Economics: Why W. S. Gosset aka "Student" Matters

Stephen T. Ziliak, Roosevelt University, United States, sziliak@roosevelt.edu
 
Suggested Citation
Stephen T. Ziliak (2014), "Balanced versus Randomized Field Experiments in Economics: Why W. S. Gosset aka "Student" Matters", Review of Behavioral Economics: Vol. 1: No. 1–2, pp 167-208. http://dx.doi.org/10.1561/105.00000008

Publication Date: 15 Jan 2014
© 2014 S. T. Ziliak
 
Subjects
Biased estimation,  Econometric theory,  Microeconometrics,  Treatment modeling,  Industrial Organization
 
Keywords
C93C9B1
Field experimentsstatistical significanceLevittList
 

Share

Login to download a free copy
In this article:
1. Introduction 
2. Recent Literature 
3. Note on Methods and Sources 
4. The First Wave Thesis 
5. Neyman Sides with Student: The Comparative Advantage of Balanced Designs 
6. Student Against the Randomization Thesis 
7. The Power and Efficiency of Balanced Designs 
8. The Third Wave Thesis and Three Easy Ways to Improve Field Experiments in Economics 
References 

Abstract

Over the past decade randomized field experiments have gained prominence in the toolkit of empirical economics and policy making. In an article titled "Field Experiments in Economics: The Past, the Present, and the Future," Levitt and List (2009) make three main claims about the history, philosophy, and future of field experiments in economics. (1) They claim that field experiments in economics began in the 1920s and 1930s in agricultural work by Neyman and Fisher. (2) They claim that artificial randomization is essential for good experimental design because, they claim, randomization is the only valid justification for Student's test of significance. (3) They claim that decision-making in private sector firms will be advanced by partnering with economists doing randomized experiments. Several areas of research have been influenced by the article despite the absence of historical and methodological review. This paper seeks to fill that gap in the literature. The power and efficiency of balanced over random designs — discovered by William S. Gosset aka Student, and confirmed by Pearson, Neyman, Jeffreys, and others adopting a balanced, decision-theoretic and/or Bayesian approach to experiments — is not mentioned in the Levitt and List article. Neglect of Student is regrettable. A body of evidence descending from Student (1911) and extending to Heckman and Vytlacil (2007) suggests that artificial randomization is neither necessary nor sufficient for improving efficiency, identifying causal relationships, and discovering economically significant differences. Three easy ways to improve field experiments are proposed and briefly illustrated.

DOI:10.1561/105.00000008