Edmonton Quality Assessment Tool for Drug Utilization Reviews: EQUATDUR-2 - The development of a scale to assess the methodological quality of a drugutilization review

OBJECTIVE. The objective of this study was to develop an instrument that wi ll assist in evaluating the methodological quality of drug utilization revi ews (DURs) and studies of prescribing appropriateness. DESIGN. An expert committee followed accepted steps for developing and test ing new instruments. Consultations on content, face validity, and scoring o f items were solicited from external experts. Seven raters tested an initia l version; subsequently, a refined instrument was designed. The Edmonton Qu ality Assessment Tool for Drug Utilization Reviews (EQUATDUR-2) evaluates 3 domains: sample selection (1 item), data collection (1 item), and data ana lysis (3 items). Sixteen raters tested EQUATDUR-2 on a random sample of DUR s. MEASURES. The study measures were reliability-using random effects intercla ss correlation coefficients for ratings by individual raters (ICC2,1) and t he mean of ratings (ICC2,k)-and variability between DUR quality levels and rater groups. RESULTS. There were significant differences in methodological quality (P < 0.001) and in mean scores comparing low-, moderate-, and high-quality DURs. Nonmethodologists' ratings exhibited significant variability (P = 0.03) an d tended to be higher. Agreement varied for individual items (ICC2,1, 0.22 to 0.44; ICC2,k, 0.81 to 0.91) and for mean summary ratings (ICC2,1, 0.42 [ 95% CI, 0.28 to 0.61]; ICC2,k, 0.92 [95% CI, 0.86 to 0.96]). The average ti me to rate each DUR was 10.0 minutes (95% CI, 9.2 to 10.9). CONCLUSIONS. EQUATDUR-2 is a succinct, self-administered instrument with ev idence of validity and reliability. We recommend that greater than or equal to 2 raters independently assess each DUR and resolve disagreements by con sensus. EQUATDUR-2 will help clinicians and decision makers to evaluate the quality of DUR studies and provide a framework for enhancing rigor in the design, conduct, and reporting of DURs.