A NEW METHOD OF ASSESSMENT OF CLINICAL TEACHING - ROC ANALYSIS

Twenty-five clinical dental students with varying clinical experience were presented with case records consisting of a clinical history and a panoramic radiograph for 25 patients with bilateral lower third mola rs ('wisdom teeth'). The students were asked to indicate how certain t hey were that each lower third molar tooth needed removal using a 6-po int rating scale. Immediately following the task, the students were pr esented with information on the indications for removal of lower third molars in the form of a lecture by a senior academic clinical teacher . One week later the students were asked to repeat the rating study, u nder the same conditions as before, using a further 25 clinical cases. Receiver operating characteristics (ROC) analysis, which provides a g raphical and quantitative assessment of a group of observers' ability to detect need for treatment, was utilized to examine differences betw een junior and senior students and between the matched pre-training an d post-training experiments. The ability of junior students to assign lower third molars for surgery was statistically no better than random selection of cases. Formal clinical teaching significantly improved t his group's performance, but had no effect on the performance of senio r students. Senior students were significantly better able correctly t o assign lower third molars for surgical intervention than junior stud ents. Therefore this study shows that clinical experience has a signif icantly greater influence on treatment-planning ability than formal te aching. ROC analysis is a useful tool for assessing the effectiveness of methods of undergraduate training.