The in-training examination in internal medicine: Resident perceptions andlack of correlation between resident scores and faculty predictions of resident performance

PURPOSE: We sought to survey residents' perceptions regarding the In-Traini ng Examination in Internal Medicine and to assess the ability of faculty me mbers to evaluate the knowledge base of internal medicine residents. SUBJECTS AND METHODS: Residents were asked about the perceived utility of t he In-Training Examination and related self-directed educational activities . Residents predicted their own performance on the examination (into upper, middle, or lower tertile). Faculty predicted housestaff's scores, and resi dents predicted the scores of interns. RESULTS: Most residents (35/36; 97%) believed that the examination was usef ul, and 91% modified their study habits or clinical rotation schedule based on its results. Approximately half of the residents accurately predicted i nto which tertile they would score. Faculty predictions of resident perform ance on the examination were accurate 49% of the time, and resident predict ions of intern scores were accurate 38% of the rime. The sensitivity of a l ower-tertile prediction by faculty was 34%, with a specificity of 90%. The sensitivity of a resident prediction of a lower-tertile intern score was 15 %, with a specificity of 98%. Both faculty and residents were more likely t o overestimate than underestimate examination scores. CONCLUSION: Residents believe that the In-Training Examination is useful an d frequently initiate educational interventions based on results. Faculty a nd residents lack the ability to evaluate accurately the knowledge of train ees that they supervise. In particular, both groups may be unable to identi fy trainees who are deficient in this element of clinical competence. Am J Med. 1999;106:206-210. (C) 1999 by Excerpta Medica, Inc.