Assessment of English proficiency in international medical graduates by physician examiners and standardized patients

Context Since 1986, the Ontario Ministry of Health has provided a medical l icensure preparation programme for international medical graduates. Because of the diversity in candidates' oral English proficiency, this competency has been viewed as a particularly important selection criterion. Objectives To assess and compare the quality of ratings of oral English pro ficiency of international medical graduates provided by physician examiners and by standardized patients (SPs). Participants and materials The study samples consisted of 73 candidates for the Ontario International Medical Graduate (IMG) Program, and physician ex aminers and SPs in five 10-minute encounter objective structured clinical e xamination (OSCE) stations. Materials used were a seven-item speaking perfo rmance rating instrument prepared for the Ontario IMG Program. Methods Rating sheets were scanned and the results analysed using SPSS 9.0 for Windows. Results Correlations between the physician and SP ratings on the seven item s ranged from 0.52 to 0.70. The SPs provided more lenient ratings. Mean alp ha reliability for the physicians' ratings on the seven items was 0.59, and for the SPs' 0.64. There was poor agreement between the two sets of raters in identifying problematic candidates. Conclusions Notwithstanding the sizable correlations between the ratings pr ovided by the two rater groups, the results demonstrated that there was lit tle agreement between the two groups in identifying the potentially problem atic candidates. The physicians were less prone than the SPs to rate candid ates as problematic. SPs may be better placed than the physician examiners to directly assess IMG candidates' oral English proficiency.