Gd. Sanders et al., Design and pilot evaluation of a system to develop computer-based site-specific practice guidelines from decision models, MED DECIS M, 20(2), 2000, pp. 145-159
Background. Local tailoring of clinical practice guidelines (CPGs) requires
experts in medicine and evidence synthesis unavailable in many practice se
ttings. The authors' computer-based system enables developers and users to
create, disseminate, and tailor CPGs, using normative decision models (DMs)
. Methods. ALCHEMIST, a web-based system, analyzes a DM, creates a CPG in t
he form of an annotated algorithm, and displays for the guideline user the
optimal strategy. ALCHEMIST'S interface enables remote users to tailor the
guideline by changing underlying input variables and observing the new anno
tated algorithm that is developed automatically. In a pilot evaluation of t
he system, a DM was used to evaluate strategies for staging non-small-cell
lung cancer. Subjects (n = 15) compared the automatically created CPG with
published guidelines for this staging and critiqued both using a previously
developed instrument to rate the CPGs' usability, accountability, and accu
racy on a scale of 0 (worst) to 2 (best), with higher scores reflecting hig
her quality. Results. The mean overall score for the ALCHEMIST GPG was 1.50
2, compared with the published-CPG score of 0.987 (p = 0.002). The ALCHEMIS
T CPG scores for usability, accountability, and accuracy were 1.683, 1.393,
and 1.430, respectively; the published CPG scores were 1.192, 0.941, and 0
.830 (each comparison p < 0.05). On a scale of 1 (worst) to 5 (best), users
' mean ratings of ALCHEMIST'S ease of use, usefulness of content, and prese
ntation format were 4.76, 3.98, and 4.64, respectively. Conclusions. The re
sults demonstrate the feasibility of a web-based system that automatically
analyzes a DM and creates a CPG as an annotated algorithm, enabling remote
users to develop site-specific CPGs. In the pilot evaluation, the ALCHEMIST
guidelines met established criteria for quality and compared favorably wit
h national CPGs. The high usability and usefulness ratings suggest that suc
h systems can be a good tool for guideline development.