MULTIREADER, MULTICASE RECEIVER OPERATING CHARACTERISTIC METHODOLOGY - A BOOTSTRAP ANALYSIS

Rationale and Objectives. We evaluated by bootstrapping the conclusion s obtained by the Dorfman-Berbaum-Metz (DBM) receiver operating charac teristic (ROC) method and by the Toledano-Gatsonis (TG) method on a we ll-known data set. Methods. We bootstrapped in two ways: resampled cas es while holding readers fixed and resampled both cases and readers. R esults. When an analysis of variance of pseudovalues implies that read er variance and all random interactions with treatment are essentially zero, then case-resampling bootstrap and the DBM and TG methods shoul d give the same results. Case-resampling bootstrap and the DBM and TG methods did give highly similar results for both individual readers an d the averages over all readers. Both the case-resampling bootstrap an d the reader-case resampling bootstrap gave smaller standard errors fo r group than for individual reader means, thereby providing evidence f or a trade-off of readers and cases with regard to precision and power in this data set. Conclusion. Case-resampling bootstrap provides some justification for the DBM and TG methods.