EVALUATION OF 2 OUTCOME PREDICTION MODELS ON AN INDEPENDENT DATABASE

Objective: To evaluate the performance of the New Simplified Acute Phy siology Score (SAPS II) and the admission Mortality Probability Model (MPM0) in a large independent database, using formal statistical asses sment. Design: Analysis of the database of a multicenter, multinationa l, prospective cohort study, EURICUS-I. Setting: Eighty nine intensive care units (ICUs) from 13 European areas. Patients: Data of 16,060 pa tients consecutively admitted to the participating ICUs were collected during a period of 4 months. Following the original SAPS II and MPM0 criteria, the analysis excluded: patients < 18 ys of age; readmissions ; patients admitted with acute myocardial infarction; burns; and patie nts in the postoperative period after coronary artery bypass surgery. All patients with a length of stay < 8 hrs were excluded from the stud y to keep comparability between both systems. A total of 10,027 patien ts were analyzed. Interventions: Collection of the first 24 hrs' admis sion data necessary for the calculation of SAPS II and MPM0 and basic demographic statistics. Vital status at discharge from the hospital wa s registered. Measurements and Main Results: Despite having a good dis criminative capability, as measured by the area under the receiver ope rating characteristic (ROC) curves (SAPS II: ROC = 0.822 +/- 0.005 SEM ; MPM0: ROC = 0.785 +/- 0.006 SEM), both models presented poor calibra tion, with significant differences between observed and predicted mort ality (Hosmer-Lemeshow goodness-of-fit tests H and C, p < .0001). Both SAPS II (predicted risk > 40%) and MPM0 (predicted risk > 30%) overes timated the risk of death. The evaluation of the uniformity of fit of SAPS II and MPM0 demonstrated large variations across the various subg roups of patients. Conclusions: The original SAPS II and MPM0 models d id not accurately predict mortality on an independent large internatio nal multicenter ICU patient database. Results of studies utilizing gen eral outcome prediction models without previous validation in the targ et population should be interpreted with prudence.