VALIDITY OF 3 CLINICAL-PERFORMANCE ASSESSMENTS OF INTERNAL-MEDICINE CLERKS

Purpose. To analyze the construct validity of three methods to assess the clinical performances of internal medicine clerks. Method. A multi trait-multimethod (MTMM) study was conducted at the Case Western Reser ve University School of Medicine to determine the convergent and diver gent validity of a clinical evaluation from (CEF) completed by faculty and residents, an objective structured clinical examination (OSCE), a nd the medicine subject test of the National Board of Medical Examiner s. Three traits were involved in the analysis: clinical skills, knowle dge, and personal characteristics. A correlation matrix was computed f or 410 third-year students who completed the clerkship between August 1988 and July 1991. Results. There was a significant (p < .01) converg ence of the four correlations that assessed the same traits by using d ifferent methods. However, the four convergent correlations were of mo derate magnitude (ranging from .29 to .47). Divergent validity was ass essed by comparing the magnitudes of the convergence correlations with the magnitudes of correlations among unrelated assessments (i.e., dif ferent traits by different methods). Seven of nine possible coefficien ts were smaller than the convergent coefficients, suggesting evidence of divergent validity. A significant CEF method effect was identified. Conclusion. There was convergent validity and some evidence of diverg ent validity with a significant method effect. The findings were simil ar for correlations corrected for attenuation. Four conclusions were r eached: (1) the reliability of the OSCE must be improved, (2) the CEF ratings must be redesigned to further discriminate among the specific traits assessed, (3) additional methods to assess personal characteris tics must be instituted, and (4) several assessment methods should be used to evaluate individual student performances.