Use of administrative data to find substandard care - Validation of the complications screening program

OBJECTIVE. The use of administrative data to identify inpatient complicatio ns is technically feasible and inexpensive but unproven as a quality measur e. Our objective was to validate whether a screening method that uses data from standard hospital discharge abstracts identifies complications of care and potential quality problems. DESIGN. This was a case-control study with structured implicit physician re views. SETTING. Acute-care hospitals in California and Connecticut in 1994. PATIENTS. The study included 1,025 Medicare beneficiaries greater than or e qual to 65 years of age. METHODS. Using administrative data, we stratified acute-care hospitals by o bserved-to-expected complication rates and randomly selected hospitals with in each state. We randomly selected cases flagged with 1 of 17 surgical com plications and 6 medical complications. We randomly selected controls from unflagged cases. MAIN OUTCOME MEASURE. Peer-review organization physicians' judgments about the presence of the flagged complication and potential quality-of-care prob lems. RESULTS. Physicians confirmed flagged complications in 68.4% of surgical an d 27.2% of medical cases. They identified potential quality problems in 29. 5% of flagged surgical and 15.7% of medical cases but in only 2.1% of surgi cal and medical controls. The rate of physician-identified potential qualit y problems among flagged cases exceeded 25% in 9 surgical screens and 1 med ical screen. Reviewers noted several potentially mitigating circumstances t hat affected their judgments about quality, including factors related to th e patients' illness, the complexity of the case, and technical difficulties that clinicians encountered. CONCLUSIONS. For some types of complications, screening administrative data may offer an efficient approach for identifying potentially problematic ca ses for physician review. Understanding the basis for physicians' judgments about quality requires more investigation,