LIMITATIONS TO THE ROBUSTNESS OF BINORMAL ROC CURVES - EFFECTS OF MODEL MISSPECIFICATION AND LOCATION OF DECISION THRESHOLDS ON BIAS, PRECISION, SIZE AND POWER
Sj. Walsh, LIMITATIONS TO THE ROBUSTNESS OF BINORMAL ROC CURVES - EFFECTS OF MODEL MISSPECIFICATION AND LOCATION OF DECISION THRESHOLDS ON BIAS, PRECISION, SIZE AND POWER, Statistics in medicine, 16(6), 1997, pp. 669-679
This paper concerns robustness of the binormal assumption for inferenc
es that pertain to the area under an ROC curve. I applied the binormal
model to rating method data sets sampled from bilogistic curves and o
bserved small biases in area estimates. Bias increased as the range of
decision thresholds decreased. The variance of area estimates also in
creased as the range of decision thresholds decreased. Together, minor
bias and inflated variance substantially altered the size and power o
f statistical tests that compared areas under bilogistic ROC curves. I
repeated the simulations by applying the binormal assumption to data
sampled from binormal curves. Biases in area estimates were minimal fo
r the binormal data, but the variance of area estimates was again high
er when the range of decision thresholds was narrow. The size of tests
that compared areas did not vary from the chosen significance level.
Power fell, however, when the variance of area estimates was inflated.
I conclude that inferences derived from the binormal assumption are s
ensitive to model misspecification and to the location of decision thr
esholds. A narrow span of decision thresholds increases the variabilit
y of area estimates and reduces the power of area comparisons. Model m
isspecification produces bias that alters test size and can exaggerate
the loss of power that accompanies increased variability. (C) 1997 by
John Wiley & Sons, Ltd.