Examining the generalizability of ratings across clerkships using a clinical evaluation form

It is necessary to average multiple clinical evaluation form (CEF) observat ions to obtain a reliable score. Combining CEF observations across clerkshi ps will provide more observations per student, bur it is unknown how an acr oss-clerkship generalization affects reliability. The authors conducted gen eralizability studies on balanced stratified random samples to examine the impact of averaging across clerkships. The study detected a student by cler kship interaction. Although the interaction magnitude was small, it had a l arge negative impact on the reliability of the mean score. The authors conc lude that averaging across clerkships for the purpose of evaluating global clinical skills will produce a less reliable measure for a fixed number of observations than averages calculated within a clerkship for evaluating cle rkship specific skills. The content specificity of clinical skills as measu red by the CEF may resemble that found using other measurement methods.