Controlling the false discovery rate in behavior genetics research

The screening of many endpoints when comparing groups from different strain s, searching for some statistically significant difference, raises the mult iple comparisons problem in its most severe form. Using the 0.05 level to d ecide which of the many endpoints' differences are statistically significan t, the probability of finding a difference to be significant even though it is not real increases far beyond 0.05. The traditional approach to this pr oblem has been to control the probability of making even one such error-the Bonferroni procedure being the most familiar procedure achieving such cont rol. However, the incurred loss of power stemming from such control led man y practitioners to neglect multiplicity control altogether. The False Disco very Rate (FDR), suggested by Benjamini and Hochberg [J Royal Stat Soc Ser B 57 (1995) 289], is a new. different, and compromising point of view regar ding the error in multiple comparisons. The FDR is the expected proportion of false discoveries among the discoveries, and controlling the FDR goes a long way towards controlling the increased error from multiplicity while lo sing less in the ability to discover real differences. In this paper we dem onstrate the problem in two studies: the study of exploratory behavior [Beh av Brain Res (2001)], and the study of the interaction of strain difference s with laboratory environment [Science 284 (1999) 1670]. We explain the FDR criterion, and present two simple procedures that control the FDR. We demo nstrate their increased power when used in the above two studies. (C) 2001 Elsevier Science B.V. All rights reserved.