Sm. Oishi et al., Using data to enhance the expert panel process - Rating indications of alcohol-related problems in older adults, INT J TE A, 17(1), 2001, pp. 125-136
Citations number
36
Categorie Soggetti
Health Care Sciences & Services
Journal title
INTERNATIONAL JOURNAL OF TECHNOLOGY ASSESSMENT IN HEALTH CARE
Objective: To enhance the validity of a well-known expert panel process, we
used data from patient surveys to identify and correct rating errors.
Methods: We used the two-round RAND/UCLA panel method to rate indications o
f harmful (presence of problems), hazardous (at risk for problems), and non
hazardous (no known risks) drinking in older adults. Results from the panel
provided guidelines for classifying older individuals as harmful, hazardou
s, or nonhazardous drinkers, using a survey. The classifications yielded un
expectedly high numbers of harmful and hazardous drinkers. We hypothesized
possible misclassifications of drinking risks and used the survey data to i
dentify indications that may have led to invalid ratings. We modified probl
ematic indications and asked three clinician panelists to evaluate the clin
ical usefulness of the modifications in a third panel round. We revised the
indications based on panelist response and reexamined drinking classificat
ions.
Results: Using the original indications, 48% of drinkers in the sample were
classified as harmful, 31% as hazardous, and 21% as nonhazardous. A review
of the indications revealed framing bias in the original rating task and v
ague definitions of certain symptoms and conditions. The modified indicatio
ns resulted in classifications of 22% harmful, 47% hazardous, and 31% nonha
zardous drinkers.
Conclusions: Analysis of survey data led to identification and correction o
f specific errors occurring during the panel-rating process. The validity o
f the RAND/UCLA method can be enhanced using data-driven modifications.