COMPARING CHECKLISTS AND DATABASES WITH PHYSICIANS RATINGS AS MEASURES OF STUDENTS HISTORY AND PHYSICAL-EXAMINATION SKILLS

Purpose. To compare two methods of rating students' performances on hi story and physical examination: (1) by using checklists completed by s tandardised patients (SPs) and databases completed by students, and (2 ) by using ratings of students by three physicians for each SP-student encounter. Method. Four cases were chosen for the study, and 30 stude nts were examined per case, The students were all in their fourth year at the Southern Illinois University School of Medicine in the spring of 1991. Two of the cases had both checklists and databases, and the r emaining two had databases only. Each SP-student encounter was videota ped and was viewed independently by three physicians unfamiliar with t he contents of the checklists and databases. The physicians' pooled ra tings were then compared with the checklist and database scores. Uncor rected and corrected correlations were obtained, with the generalizabi lity coefficient used as the index of reliability. Results. Interrater generalizability of physicians' ratings was very good, ranging from . 65 to .93 for overall ratings. Generalizability of physicians' ratings pooled across the four cases was .85. Checklist scores tended to corr elate higher with physicians' ratings than did database scores: across the cases, correlation coefficients between physicians' ratings and c hecklist scores and database scores were .65 and .39, respectively. Co nclusion. The checklist scores correlated strongly with the physicians ' ratings of history and physical-examination skills, providing some e vidence of validity for their use. The checklist scores correlated muc h better with the physicians' ratings than did the database scores. Po ssible explanations for this finding are discussed.