Prognostication and certainty in the pediatric intensive care unit

Objective. Prognostication is central to developing treatment plans and rel aying information to patients, family members, and other health care provid ers. The degree of confidence or certainty that a health care provider has in his or her mortality risk assessment is also important, because a provid er may deliver care differently depending on their assuredness in the asses sment. We assessed the performance of nurse and physician mortality risk es timates with and without weighting the estimates with their respective degr ees of certainty. Methods. Subjective mortality risk estimates from critical care attendings (n = 5), critical care fellows (n = 9), pediatric residents (n = 34), and n urses (n = 52) were prospectively collected on at least 94% of 642 eligible , consecutive admissions to a tertiary pediatric intensive care unit (PICU) . A measure of certainty (continuous scale from 0 to 5) accompanied each mo rtality estimate. Estimates were evaluated with 2 x 2 outcome probabilities , the kappa statistic, the area under the receiver operating characteristic s curve, and the Hosmer and Lemeshow goodness-of-fit chi(2) statistic. The estimates were then reevaluated after weighting predictions by their respec tive degree of certainty. Results. Overall, there was a significant difference in the predictive accu racy between groups. The mean mortality predictions from the attendings (6. 09%) more closely approximated the true mortality rate (36 deaths, 5.61%) w hereas fellows (7.87%), residents (10.00%), and nurses (16.29%) overestimat ed the mean overall PICU mortality. Attendings were more certain of their p redictions (4.27) than the fellows (4.01), nurses (3.79), and residents (3. 75). All groups discriminated well (area under receiver operating character istics curve range, 0.86-0.93). Only PICU attendings and fellows did not si gnificantly differ from ideal calibration ( x 2). When mortality prediction s were weighted with their respective certainties, their performance improv ed. Conclusions. The level of medical training correlated with the provider's a bility to predict mortality risk. The higher the level of certainty associa ted with the mortality prediction, the more accurate the prediction; howeve r, high levels of certainty did not guarantee accurate predictions. Measure s of certainty should be considered when assessing the performance of morta lity risk estimates or other subjective outcome predictions.