Community-wide assessment of intensive care outcomes using a physiologically based prognostic measure - Implications for critical care delivery from Cleveland Health Quality Choice

Study objectives: To examine the applicability of a previously developed in tensive care prognostic measure to a community-based sample of hospitals, a nd assess Variations in severity-adjusted mortality across a major metropol itan region. Design: Retrospective cohort study. Setting: Twenty-eight hospitals with 38 ICUs participating in a community-w ide initiative to measure performance supported by the business community, hospitals, and physicians. Patients: Included in the study were 116,340 consecutive eligible patients admitted to medical, surgical, neurologic, and mixed medical/surgical ICUs between March 1, 1991, and March 31, 1995. Main outcome measures: The risk of hospital mortality was assessed using a previous risk prediction equation that was developed in a national sample, and a reestimated logistic regression model fit to the current sample. The standardized mortality ratio (SMR) (actual/predicted mortality) was used to describe hospital performance. Results: Although discrimination of the previous national risk equation in the current sample was high (receiver operating characteristic [ROC] curve area 0.90), the equation systematically overestimated the risk of death and was not as well calibrated (Hosmer-Lemeshow statistic, 2407.6, 8 df, p < 0 .001). The locally derived equation had similar discrimination (ROC curve a rea = 0.91), but had improved calibration across all ranges of severity (Ho smer-Lemeshow statistic = 13.5, 8 df,p = 0.10), Hospital SMRs ranged from 0 .85 to 1.21, and four hospitals had SMRs that were higher or lower (p < 0.0 1) than 1.0. Variation in SMRs tended to be greatest during the first year of data collection. SMRs also tended to decline over the 4 years (1.06, 1.0 2, 0.98, and 0.94 in years 1 to 4, respectively), as did mean hospital leng th of stay (13.0, 12.4, 11.6, and 11.1 days in years 1 to 4; p < 0.001). Ho wever, excluding the increasing (p < 0.001) number of patients discharged t o skilled nursing facilities attenuated much of the decline in standardized mortality over time. Conclusions: A previously validated physiologically based prognostic measur e successfully stratified patients in a large community-based sample by the ir risk of death. However, such methods may require recalibration when appl ied to new samples and to reflect changes in practice over time. Moreover, although significant variations in hospital standardized mortality were obs erved, changing hospital discharge practices suggest that in-hospital morta lity may no longer be an adequate measure of ICU performance. Community-wid e efforts with broad-based support from business, hospitals, and physicians can be sustained over time to assess outcomes associated with ICU care. Su ch efforts may provide important information about variations in patient ou tcomes and changes in practice patterns over time. Future efforts should as sess the impact of such community-wide initiatives on health-care purchasin g and institutional quality improvement programs.