VALIDATING THE STANDARDIZED-PATIENT ASSESSMENT ADMINISTERED TO MEDICAL-STUDENTS IN THE NEW-YORK-CITY CONSORTIUM

Purpose. To test the criterion validity of existing standardized-patie nt (SP)-examination scores using global ratings by a panel of faculty- physician observers as the gold-standard criterion; to determine wheth er such ratings can provide a reliable gold-standard criterion to be u sed for validity-related research; and to encourage the use of these g old standard ratings for validation research and examination developme nt, including scoring and standard setting, and for enhancing understa nding of the clinical competence construct. Method. Five faculty physi cians independently observed and rated videotaped performances of 44 s tudents from one medical school on the seven SP cases that make up the fourth-year assessment administered at The Morchand Center of Mount S inai School of Medicine to students in the eight member schools in the New York City Consortium. Results. The validity coefficients showed c orrelations between scores on the examination and the overall ratings ranging from .60 to .70. The reliability coefficients for ratings of o verall examination performance reached the commonly recommended .80 le vel and were very close at the case lever, with interrater reliabiliti es generally in the .70 to .80 range. Conclusion. The results are enco uraging, with validity coefficients high enough to warrant optimism ab out the possibility of increasing them to the recommended .80 level, b ased on further studies to identify those measurable performance chara cteristics that most reflect the gold-standard ratings. The high inter rater reliabilities indicate that faculty-physician ratings of perform ance on SP cases and examinations can or may be able to provide a reli able gold standard for validating and refining SP assessment.