Objectives This study was designed to describe the variation in marking ten
dencies among different examiners in an oral examination.
Design Marks awarded in a family practice board examination between 1984 an
d 1996 were analysed, relating to 5328 examination sessions graded by 94 ex
aminers. Examiners were ranked by the rates at which they awarded 'fail', '
pass' or 'distinction' grades. The effects of examiners' gender, experience
, academic rank, regional affiliation and country of qualification on exami
ner behaviour were studied.
Setting National Family Medicine Examination Board, Scientific Council, Isr
ael Medical Association.
Subjects Oral examiners.
Results Eighteen per cent of examiners were classified as 'tough', being in
the lowest tertile for 'distinction' rates and the highest tertile for 'fa
ilure' rates; 19% were classified as 'mild'; 52% were 'regular', falling in
the middle tertile for both distinction and failure rates. Four per cent o
f examiners were in the top tertile for both distinctions and failures, lab
elled 'extremists', and 6% were in the bottom tertile for both, and were la
belled 'noncommittal'. Higher failure rates were associated with examiners'
academic rank, experience and graduation from an English-speaking medical
school.
Conclusions Examiners differ significantly in their degree of severity. Tho
se who demonstrate clearly deviant patterns of grading should be withdrawn.
Candidates should be presented with a balanced panel of examiners, and a d
egree of standardization of content should be introduced into oral examinat
ions.