RELIABILITY OF PERFORMANCE-BASED CLINICAL SKILL ASSESSMENT OF EMERGENCY-MEDICINE RESIDENTS

Objective: To test the overall reliability of a performance-based clin ical skill assessment for entering emergency medicine (EM) residents. Also, to investigate the reliability of separate reporting of diagnost ic and management scores for a standardized patient case, subjective s coring of patient notes, and interstation exercise scores. Methods: Th irty-four first-year EM residents were tested using a 10-station stand ardized patient (SP) examination. Following each 10-minute encounter, the residents completed a patient note that included differential diag nosis and management. The residents also were asked to read an ECG or chest x-ray (CXR) associated with each case. History, physical examina tion, and interpersonal skills were scored by the SPs. The patient not e, CXR, and ECG readings were scored by faculty emergency physicians, Intercase reliability was determined for the residents. Results: Globa l score reliability, Cronbach's alpha = 0.85. Reliabilities for the ot her components were: history, 0.77; physical examination, 0.83; and in terpersonal skills, 0.80. Differential diagnosis and management reliab ilities were 0.61 and 0.66, respectively. Subjective scoring of the pa tient note resulted in acceptable reliability for legibility (0.80), h istory completeness (0.80), and history organization (0.81). Physical examination completeness and organization reliabilities were 0.74 and 0.73. For ECG and CXR readings, alpha = 0.74 and 0.34, respectively. C onclusions: SPs can be used to reliably assess bedside clinical skills of EM residents. While component reliability levels are slightly lowe r than the global clinical skill reliability coefficient, they are sti ll high enough to use for identification of individual strengths and w eaknesses.