Cd. Kreiter et al., A generalizability study of a new standardized rating form used to evaluate students' clinical clerkship performances, ACAD MED, 73(12), 1998, pp. 1294-1298
Purpose. To investigate the measurement characteristics of standardized cli
nical evaluation forms (CEFs) used to assign grades for clerkship performan
ce.
Method. In 1996-97, the authors reviewed 5,168 CEFs completed for 175 stude
nts in eight clerkships. Limiting their analysis to the three clerkships th
at produced the most CEFs, the authors conducted a generalizability study t
o determine the five variance components for each clerkship. A decision stu
dy then calculated the generalizability coefficients and standard errors of
measurement in each clerkship for varied numbers of raters and CEF items.
Results. The generalizability study found large variance components attribu
table to rater and rating context. The decision study found that, when thre
e or more raters completed CEFs for a student, the generalizability coeffic
ient and standard error of measurement reached levels acceptable for gradin
g. Increasing the number of items on the CEF had no significant effect.
Conclusion. The reliability of assigning students clerkship grades based on
single CEFs is unacceptably low. However, CEFs can accurately measure stud
ents' clerkship performances if completed by three or more raters.