Randomized Experiments as the Bronze Standard
Richard A. Berk at UCLA Stats dept published a paper entitled Randomized Experiments as the Bronze Standard
The paper is a thorough analysis of the pros and cons — mostly the cons — of randomized experiments, although I admit I haven’t read it thoroughly.ABSTRACT:
In this paper, the strengths and weakness of randomized field ex- periments are discussed. Although it seems to be common knowledge that random assignment balances experimental and control groups on all confounders, other features of randomized field experiments are somewhat less appreciated. These include the role of random assign- ment in statistical inference and representations of the mechanisms by which the treatment has its impact. Randomized experiments also have important limitations and are sub ject to the fidelity with which they are implemented. In the end, randomized field experiments are still the best way to estimate causal effects, but are a considerable distance from perfection.SUGGESTED CITATION:
Richard Berk, "Randomized Experiments as the Bronze Standard" (August 2, 2005). Department of Statistics, UCLA. Department of Statistics Papers. Paper 2005080201.
http://repositories.cdlib.org/uclastat/papers/2005080201
The "gold standard" reference is well known in the medical context, but the meme has infested the field of education in the past decade. Berk’s message is clear — there is no gold standard or even silver standard; randomized experiments should not be the default or ideal methodology. I have little to add excpet to say that, in education, randomized assignment is for the most part the emperor’s new cloth.