DEGENERACY AND DISCRETE RECEIVER OPERATING CHARACTERISTIC RATING DATA

Rationale and Objectives. Observer performance studies sometimes use t oo few cases for estimating diagnostic accuracy from binormal receiver operating characteristic (ROC) curves, One important problem is degen erate data sets. We compared a new algorithm, RSCORE4, with the exact- solution approach to degeneracy in ROCFIT and with the Wilcoxon statis tic. Methods. Degenerate ROC solutions result from empty cells in the data matrix. We addressed this problem by adding a small constant to e mpty cells in a maximum-likelihood program, RSCORE4, When this method failed, the program branched to a pattern-search algorithm. We tested the program in a series of Monte Carlo studies. Results. RSCORE4 conve rged to nondegenerate solutions in every case and gave results closer to population values than ROCFIT or Wilcoxon. ROCFIT converged to exac t-fit degenerate solutions, those with zero or infinite parameter valu es, in more than 40% of the samples. The Wilcoxon statistic was biased . Conclusion. RSCORE4 seems to outperform other currently recommended methods for dealing with degeneracy.