MEASURING THE CLINICAL CONSISTENCY OF PANELISTS APPROPRIATENESS RATINGS - THE CASE OF CORONARY-ARTERY BYPASS-SURGERY

Objective: To assess the clinical consistency of expert panelists' rat ings of appropriateness for coronary artery bypass surgery. Design: Qu antitative analysis of panelists' ratings. Participants: Nine physicia ns (three cardiothoracic surgeons, four cardiologists, and two interni sts) convened by RAND to establish criteria for the appropriateness of coronary artery bypass surgery. Main outcomes measures: Percentage of indication-pairs given clinically inconsistent ratings (i.e. higher r ating assigned to one member of an indication-pair when rating should have been equal or lower). Results: In the final round of appropriaten ess ratings, among 1785 pairs of indications differing only on a singl e clinical factor (e.g.: three-vessel vs. two-vessel stenosis), 6.6% w ere assigned clinically inconsistent ratings by individual panelists, but only 2.7% received inconsistent ratings from the panel as a whole (using the median panel rating as the criterion). Internists on the pa nel provided fewer inconsistent ratings (4.6%) than either cardiologis ts (7.8%) or cardiothoracic surgeons (6.3%) (p<0.001). More inconsiste ncies were noted when the factor distinguishing otherwise identical in dications was symptom severity (inconsistency rate? 13.2%) or intensit y of medical therapy (13.2%) than when it was number of stenosed vesse ls (3.8%) or proximal left anterior descending (PLAD) involvement (1.9 %). Contrary to expectations, panelists' inconsistency rates increased between the initial and final rounds of appropriateness ratings (from 3.9 to 6.6%, p < 0.001). Panelists' mean ratings across indications w ere only weakly correlated with individual inconsistency rates (r = 0. 18, p = ns). Conclusions: The RAND/UCLA method for assessing the appro priateness of coronary revascularization generally produces criteria t hat are clinically consistent. However, research is needed to understa nd the sources of panelists' inconsistencies and to reduce inconsisten cy rates further. (C) 1997 Elsevier Science Ireland Ltd.