Y. Benjamini et Y. Hochberg, On the adaptive control of the false discovery fate in multiple testing with independent statistics, J ED BEH ST, 25(1), 2000, pp. 60-83
A new approach to problems of multiple significance testing was presented i
n Benjamini and Hochberg (1995), which calls for controlling the expected r
atio of the number of erroneous rejections to the number of rejections-the
False Discovery Rate (FDR). The procedure given there was shown to control
the FDR for independent test statistics. When some of the hypotheses are in
fact false, that procedure is too conservative. Ne present here an adaptiv
e procedure, where the number of true null hypotheses is estimated first as
in Hochberg and Benjamini (1990), and this estimate is used in the procedu
re of Benjamini and Hochberg (1995). The result is still a simple stepwise
procedure, to which we also give a graphical companion. The new procedure i
s used in several examples drawn from educational and behavioral studies, a
ddressing problems in multi-center studies, subset analysis and meta-analys
is. The examples vary in the number of hypotheses tested, and the implicati
on of the new procedure on the conclusions. III a large simulation study of
independent test statistics the adaptive procedure is shown to control the
FDR and have substantially better power than the previously suggested FDR
controlling method, which by itself is more powerful than the traditional f
amilywise error-rate controlling methods. In cases where most of the tested
hypotheses are far from being true there is hardly any penalty due to the
simultaneous testing of many hypotheses.