Limitations of the facility-specific standardized mortality ratio for profiling health care quality in dialysis

Health care quality is assessed by profiling measures of care and/or health outcomes. However, such tools to measure outcome as standardized mortality ratio (SMR) are often used without thorough validation of their strengths and limitations. Our study compared the dialysis facility-specific SMR and SMR-based rating using different statistical methods and followed them over time, All Fresenius Medical Care, North America dialysis facilities (n = 3 77) that contributed patient data from 1993 to 1995 (>103,500 patient-years ) were included, Four distinct statistical methods (US Renal Data System [U SRDS], Poisson, logistic, and Cox regression) were used to compute facility -specific SMRs and rank and classify facilities. The analysis compared the SMR and SMR-based rating of dialysis facilities between SMR method and over time. Different methods produced statistically significant differences in SMR distribution (P < 0.05). The USRDS method produced SMR values that decr eased over time (P < 0.001). Based on 90% confidence intervals to determine outliers, the SMR-dependent ranking of dialysis facilities varied by metho d (P < 0.001). SMR-based ranking was stable over time except for the USRDS method (P < 0.001). Contingency table analysis showed up to a 33% total mis classification rate between SMR methods when ranking facilities. The facili ty-specific SMR and SMR-based ranking are both sensitive to statistical tec hnique, Because the SMR yields different results in a year and over time an d because there is no demonstrable gold standard, conclusions based on any one technique are unstable and unreliable. Regulatory monitoring, actions, and/or performance awards should be avoided based on this measure. However, a facility-specific SMR estimated in any valid way may be useful as an epi demiological research tool, (C) 2001 by the National Kidney Foundation, Inc .