LIMITATIONS TO THE ROBUSTNESS OF BINORMAL ROC CURVES - EFFECTS OF MODEL MISSPECIFICATION AND LOCATION OF DECISION THRESHOLDS ON BIAS, PRECISION, SIZE AND POWER

This paper concerns robustness of the binormal assumption for inferenc es that pertain to the area under an ROC curve. I applied the binormal model to rating method data sets sampled from bilogistic curves and o bserved small biases in area estimates. Bias increased as the range of decision thresholds decreased. The variance of area estimates also in creased as the range of decision thresholds decreased. Together, minor bias and inflated variance substantially altered the size and power o f statistical tests that compared areas under bilogistic ROC curves. I repeated the simulations by applying the binormal assumption to data sampled from binormal curves. Biases in area estimates were minimal fo r the binormal data, but the variance of area estimates was again high er when the range of decision thresholds was narrow. The size of tests that compared areas did not vary from the chosen significance level. Power fell, however, when the variance of area estimates was inflated. I conclude that inferences derived from the binormal assumption are s ensitive to model misspecification and to the location of decision thr esholds. A narrow span of decision thresholds increases the variabilit y of area estimates and reduces the power of area comparisons. Model m isspecification produces bias that alters test size and can exaggerate the loss of power that accompanies increased variability. (C) 1997 by John Wiley & Sons, Ltd.