A contaminated binormal model for ROC data - Part I. Some interesting examples of binormal degeneracy

Rationale and Objectives. Receiver operating characteristic (ROC) data with false-positive fractions of 0 are often difficult to fit with standard ROC methods and are sometimes discarded. Some extreme examples of such data we re analyzed to evaluate the nature of these difficulties. Materials and Methods. Rating reports of fracture for single-view ankle rad iographs were analyzed with the binormal ROC model and with two ROC models that keep the ROC curve from crossing the chance line. Because fractures we re almost never reported that were not present, some views and locations yi elded only ROC points with false-positive fractions of 0, while others yiel ded at least one ROC point with a non-0 false-positive fraction. Results. The models tended to yield ROC areas close to or equal to 1. ROC a reas of 1 imply a true-positive fraction close to 1; yet the data contained no such fractions. When all false-positive fractions were 0, the true-posi tive fraction could be much higher for one view than another for all observ ers. ROC areas gave little or no hint of these unmistakable differences in performance. Conclusion. These data challenge the validity and robustness of current ROC models. A key aspect of ankle fractures is that some may be visible on one view but not at all visible on another.