In a randomized experiment, the investigator creates a clear and relatively
unambiguous comparison of treatment groups by exerting tight control over
the assignment of treatments to experimental subjects, ensuring that compar
able subjects receive alternative treatments. In an observational study, th
e investigator lacks control of treatment assignments and must seek a clear
comparison in other ways. Care in the choice of circumstances in which the
study is conducted can greatly influence the quality of the evidence about
treatment effects. This is illustrated in detail using three observational
studies that use choice effectively, one each from economics, clinical psy
chology and epidemiology. Other studies are discussed more briefly to illus
trate specific points. The design choices include (i) the choice of researc
h hypothesis, (ii) the choice of treated and control groups, (iii) the expl
icit use of competing theories, rather than merely null and alternative hyp
otheses, (iv) the use of internal replication in the form of multiple manip
ulations of a single dose of treatment, (v) the use of undelivered doses in
control groups, (vi) design choices to minimize the need for stability ana
lyses, (vii) the duration of treatment and (viii) the use of natural blocks
.