Combining physician's subjective and physiology-based objective mortality risk predictions

Objective: None of the currently available physiology-based mortality risk prediction models incorporate subjective judgements of healthcare professio nals, a source of additional information that could improve predictor perfo rmance and make such systems more acceptable to healthcare professionals. T his study compared the performance of subjective mortality estimates by phy sicians and nurses with a physiology-based method, the Pediatric Risk of Mo rtality (PRISM) III, Then, healthcare provider estimates were combined with PRISM III estimates using Bayesian statistics. The performance of the Baye sian model was then compared with the original two predictions. Design: Concurrent cohort study. Setting: A tertiary pediatric intensive care unit at a university affiliate d children's hospital. Patients: Consecutive admissions to the pediatric intensive care unit. Interventions: None. Measurements and Main Results: For each of the 642 consecutive eligible pat ients, an exact mortality estimate and the degree of certainty (continuous scale from 1 to 5) associated with the estimate was collected from the atte nding, fellow, resident, and nurse responsible for the patient's care. Baye sian statistics were used to combine the PRISM III and certainty weighted s ubjective predictions to create a third Bayesian estimate of mortality. PRI SM III discriminated survivors from nonsurvivors very well (area under curv e [AUC], 0.924) as did the physicians and nurses (AUCs attendings, 0.953; f ellows, 0.870; residents, 0.923; nurses, 0.935). Although the AUCs of the h ealthcare providers were not significantly different from the AUCs of PRISM III, the Bayesian AUCs were higher than both the healthcare providers' AUC s (p less than or equal to .09 for all) and PRISM III AUCs. Similarly, the calibration statistics for the Bayesian estimates were superior to the cali bration statistics for both the healthcare providers and PRISM III models. Conclusions: The results of this study demonstrated that healthcare provide rs' subjective mortality predictions and PRISM III mortality predictions pe rform equally well. The Bayesian model that combined provider and PRISM III mortality predictions was more accurate than either provider or PRISM III alone and may be more acceptable to physicians. A methodology using subject ive outcome predictions could be more relevant to individual patient decisi on support.