Jw. Peabody et al., Comparison of vignettes, standardized patients, and chart abstraction - A prospective validation study of 3 methods for measuring quality, J AM MED A, 283(13), 2000, pp. 1715-1722
Citations number
61
Categorie Soggetti
General & Internal Medicine","Medical Research General Topics
Context Better health care quality is a universal goal, yet measuring quali
ty has proven to be difficult and problematic. A central problem has been i
solating physician practices from other effects of the health care system,
Objective To validate clinical vignettes as a method for measuring the comp
etence of physicians and the quality of their actual practice.
Design Prospective trial conducted in 1997 comparing 3 methods for measurin
g the quality of care for 4 common outpatient conditions: (1) structured re
ports by standardized patients (SPs), trained actors who presented unannoun
ced to physicians' clinics (the gold standard); (2) abstraction of medical
records for those same visits; and (3) physicians' responses to clinical vi
gnettes that exactly corresponded to the SPs' presentations.
Setting Outpatient primary care clinics at 2 Veterans Affairs medical cente
rs.
Participants Ninety-eight (97%) of 101 general internal medicine staff phys
icians, faculty, and second- and third-year residents consented to be rando
mized for the study. From this group, 10 physicians at each site were rando
mly selected for inclusion.
Main Outcome Measures A total of 160 quality scores (8 cases x 20 physician
s) were generated for each method using identical explicit criteria based o
n national guidelines and local expert panels, Scores were defined as the p
ercentage of process criteria correctly met and were compared among the 3 m
ethods.
Results The quality of care, as measured by all 3 methods, ranged from 76.2
% (SPs) to 71.0% (vignettes) to 65.6% (chart abstraction). Measuring qualit
y using vignettes consistently produced scores closer to the gold standard
of SP scores than using chart abstraction, This pattern was robust when the
scores were disaggregated by the 4 conditions (P<.001 to <.05), by case co
mplexity (P<.001), by site (P<.001), and by level of physician training (P
values from <.001 to <.05). The pattern persisted, although less dominantly
, when we assessed the component domains of the clinical encounter-history,
physical examination, diagnosis, and treatment, Vignettes were responsive
to expected directions of variation in quality between sites and levels of
training. The vignette responses did not appear to be sensitive to physicia
ns' having seen an SP presenting with the same case.
Conclusions Our data indicate that quality of health care can be measured i
n an outpatient setting by using clinical vignettes. Vignettes appear to be
a valid and comprehensive method that directly focuses on the process of c
are provided in actual clinical practice. Vignettes show promise as an inex
pensive case-mix adjusted method for measuring the quality of care provided
by a group of physicians.