AN OBJECTIVE STRUCTURED CLINICAL EXAMINATION FOR EVALUATING PSYCHIATRIC CLINICAL CLERKS

Purpose. To assess the feasibility, reliability, and validity of an ob jective structured clinical examination (OSCE) for psychiatric clinica l clerks. Method. In 1995 two parallel forms of a ten-station OSCE (ei ght clinical stations, two writing stations) were developed at the Uni versity of Toronto Faculty of Medicine. Each 12-minute performance-bas ed clinical station was assessed by a faculty psychiatrist using both a checklist for each student's performance content and a global-rating scale of the performance process. The students' clinical-station scor es were calculated as the average of their content and process scores (expressed as percentages). Examiners also recorded an overall judgmen t of each student's performance (pass, borderline, or fail) and wrote [in collaboration with the standardized patient (SP) at that station] comments on each student's performance. There were two criteria for a passing grade: a total mark of 60% or higher across all ten stations a nd a ''pass'' or ''borderline'' mark in at least five of the eight cli nical stations. Each OSCE form was administered three times. Results. The first form was used to examine 94 clerks, the second form to exami ne 98 clerks. The students' mean scores for the two forms were 70.47% (SD, 6.33%) and 67.66% (SD, 7.05%), respectively. In addition to the s tandard evaluation information collected on the students, several crit ical incidents occurred (e.g., a student's loss;of control of emotions ) that may identify potential problems in professional conduct. The di rect cost for one administration of the examination was approximately Can$3,300: the largest portion of this was for the SPs' time spent in training and performing their roles. Conclusion. Preliminary evidence suggests that a psychiatry OSCE is feasible for assessing complex psyc hiatric skills. However, careful attention must be paid to SP training , examination monitoring, detection of critical incidents, and provisi on of feedback to students, faculty, and SPs. The university's previou s system of oral examinations required approximately 600 faculty hours per year. The OSCE requires approximately 450 faculty hours, and the 150 hours saved almost cover the Can$20,000 that the examination costs each year. In all, the OSCE is an evaluation system that has demonstr able reliability and is more enjoyable for both the faculty and the st udents.