Developing case-specific checklists for standardized-patient-based assessments in internal medicine: A review of the literature

Purpose. To review the literature on the methods used in writing case-speci fic checklists for studies of internal medicine physicians' performances th at were assessed by standardized patients. Method. The authors searched Medline, Embase, Psychlit, and ERIC for articl es in English published between 1966 and February 1998. The following searc h string was used: "[(standardi" or simulat" or programm") near (patient* o r client" or consultati")] and internal medicine." The authors then searche d the reference lists of papers retrieved from the database searches, as we ll as those from seven proceedings of the International Ottawa Conference o n Medical Education and Assessment. Results. The procedure yielded 29 relevant articles: database searches yiel ded 14 published reports dealing with case-specific checklists, 11 articles were culled from the reference lists of these papers, and the Ottawa Confe rence proceedings yielded four articles, Only 12 articles reported specific ally on the development of checklists. In general, there were three sources used for developing checklists: panels of experts, the investigators thems elves, and responses from expert physicians to written protocols. No articl e indicated that researchers had relied exclusively on data from the litera ture to compose their checklists. Only three articles indicated that litera ture sources had informed their checklist development. All articles except one relied on explicit criteria for the inclusion of items on the checklist s. In 21 of the 29 articles, the checklists had been scored by SPs, but the scoring of specific items on the checklists varied according to the purpos e of the SP-physician encounter. Only four of the articles made the checkli sts available or indicated that the checklists could be obtained from the a uthors. Conclusion. The development of case-specific checklists for SP examinations of physicians' performance has received little attention. To judge the val idity of studies of physicians' performances that: use SPs, the development processes for the checklists need to be more fully described to enable rea ders to evaluate the validity and reliability of the studies.