Prediction bias and selection bias: An empirical analysis

Empirical examination of the relation between prediction bias and selection bias has been hampered by the lack of a valid definition of selection bias . In this article, we show that such a definition is possible in the specia l case of fixed selection, where the only reason for rejecting otherwise ac ceptable applicants is that their number exceeds the number of available pl aces. Using the proposed definition, the empirical relation between predict ion bias and selection bias with respect to prominent social groups is exam ined. The results indicate that although the 2 biases are related, the rela tion is not isomorphic: First, it is mediated by the selection ratio; secon d, for most selection ratios, its strength is only moderate. Thus, both the direction and the magnitude of prediction bias may be misleading as a basi s for determining the corresponding direction and magnitude of selection bi as. In particular, lack of prediction bias does not entail lack of selectio n bias. Hence, examination of selection bias with respect to social groups cannot rely only on empirical investigation of prediction bias. Rather, it calls for direct investigation of selection bias per se.