Rationale and Objectives. We evaluated by bootstrapping the conclusion
s obtained by the Dorfman-Berbaum-Metz (DBM) receiver operating charac
teristic (ROC) method and by the Toledano-Gatsonis (TG) method on a we
ll-known data set. Methods. We bootstrapped in two ways: resampled cas
es while holding readers fixed and resampled both cases and readers. R
esults. When an analysis of variance of pseudovalues implies that read
er variance and all random interactions with treatment are essentially
zero, then case-resampling bootstrap and the DBM and TG methods shoul
d give the same results. Case-resampling bootstrap and the DBM and TG
methods did give highly similar results for both individual readers an
d the averages over all readers. Both the case-resampling bootstrap an
d the reader-case resampling bootstrap gave smaller standard errors fo
r group than for individual reader means, thereby providing evidence f
or a trade-off of readers and cases with regard to precision and power
in this data set. Conclusion. Case-resampling bootstrap provides some
justification for the DBM and TG methods.