Measuring performance in chest radiography

PURPOSE: To use a standardized set of chest radiographs to quantify interob server differences and to provide a basis for comparing the diagnostic perf ormance of physicians. MATERIALS AND METHODS: A standardized set of 60 chest radiographs was prese nted to 162 study participants. Each participant reviewed the radiographs a nd recorded his or her diagnostic impression by using a fixed five-point sc ale. These response data were used to generate receiver operating character istic curves and to establish performance benchmarks. The variations in per formance were tested for statistical significance. RESULTS: Significant interobserver variability was identified during these assessments. The composite group of board-certified radiologists demonstrat ed performance superior to that of the radiology residents and nonradiologi st physicians. CONCLUSION: By using a receiver operating characteristic approach and a sta ndardized set of chest radiographs, observer accuracy and variability are e asily quantified. This approach provides a basis for comparing the diagnost ic performance of physicians. When value is measured as a diminution in unc ertainty, board-certified radiologists contribute substantial value to the diagnostic imaging system.