A. Agresti et B. Caffo, Simple and effective confidence intervals for proportions and differences of proportions result from adding two successes and two failures, AM STATISTN, 54(4), 2000, pp. 280-288
The standard confidence intervals for proportions and their differences use
d in introductory statistics courses have poor performance, the actual cove
rage probability often being much lower than intended. However, simple adju
stments of these intervals based on adding four pseudo observations, half o
f each type, perform surprisingly well even for small samples. To illustrat
e, for a broad variety of parameter settings with 10 observations in each s
ample, a nominal 95% interval for the difference of proportions has actual
coverage probability below .93 in 88% of the cases with the standard interv
al but in only 1% with the adjusted interval; the mean distance between the
nominal and actual coverage probabilities is .06 for the standard interval
, but .01 for the adjusted one. In teaching with these adjusted intervals,
one can bypass awkward sample size guidelines and use the same formulas wit
h small and large samples.