Substitution and dropout bias in social experiments: A study of an influential social experiment

This paper considers the interpretation of evidence from social experiments when persons randomized out of a program being evaluated have good substit utes for it, and when persons randomized into a program drop out to pursue better alternatives. Using data from an experimental evaluation of a classr oom training program, we document the empirical importance of control group substitution and treatment group dropping out. Evidence that one program i s ineffective relative to close substitutes is not evidence that the type o f service provided by all of the programs is ineffective, although that is the way experimental evidence is often interpreted.