Prevalence-value-accuracy plots: A new method for comparing diagnostic tests based on misclassification costs

The clinical accuracy of diagnostic tests commonly is assessed by ROC analy sis. ROC plots, however, do not directly incorporate the effect of prevalen ce or the value of the possible test outcomes on test performance, which ar e two important factors in the practical utility of a diagnostic test. We d escribe a new graphical method, referred to as a prevalence-value-accuracy (PVA) plot analysis, which includes, in addition to accuracy, the effect of prevalence and the cost of misclassifications (false positives and false n egatives) in the comparison of diagnostic test performance. PVA plots are c ontour plots that display the minimum cost attributable to misclassificatio ns (z-axis) at various optimum decision thresholds over a range of possible values for prevalence (x-axis) and the unit cost ratio (UCR; y-axis), whic h is an index of the cost of a false-positive vs a false-negative test resu lt. Another index based on the cost of misclassifications can be derived fr om PVA plots for the quantitative comparison of test performance. Depending on the region of the PVA plot that is used to calculate the misclassificat ion cost index, it can potentially lead to a different interpretation than the ROC area index on the relative value of different tests. A PVA-threshol d plot, which is a variation of a PVA plot, is also described for readily i dentifying the optimum decision threshold at any given prevalence and UCR. In summary, the advantages of PVA plot analysis are the following: (a) it d irectly incorporates the effect of prevalence and misclassification costs i n the analysis of test performance; (b) it yields a quantitative index base d on the costs of misclassifications for comparing diagnostic tests; (c) it provides a way to restrict the comparison of diagnostic test performance t o a clinically relevant range of prevalence and UCR; and (d) it can be used to directly identify an optimum decision threshold based on prevalence and misclassification costs. (C) 1999 American Association for Clinical Chemis try.