Information collected during the residency match process does not predict clinical performance

Objective: To determine whether information collected during the National R esident Matching Program (NRMP) predicts clinical performance during reside ncy. Methods: Ten faculty members rated the overall quality of 69 pediatric hous e officers as clinicians. After rating by the faculty, folders were reviewe d for absolute rank on the NRMP match list; relative ranking (where they ra nked in their postgraduate year 1 [PGY-1] group); scores on part I of the N ational Board of Medical Examiners (NBME) examination; grades during medica l school pediatrics and internal medicine rotations; membership in the Alph a Omega Alpha Medical Honor Society; scores of faculty interviews during in tern application; scores on the pediatric in-service examination during PGY -1; and scores on the American Board of Pediatrics certification examinatio n. Results: There was substantial agreement among faculty raters as to the ove rall quality of the residents (agreement rate, 0.60; kappa = 0.50; P = .001 ). There was little correlation between faculty ratings and absolute (r = 0 .19; P = .11) or relative (r = 0.20; P = .09) ranking on the NRMP match lis t. Individuals ranked in the top 10 of the match list had higher faculty ra tings than did their peers (mean +/- SD, 3.66 +/- 1.22 vs 3.0 +/- 1.27; P = .03), as did individuals ranked highest in their PGY-1 group (mean +/- SD, 3.88 +/- 1.45 vs 3.04 +/- 1.24; P = .03). There was no correlation between faculty ratings and scores on part I of the NBME examination (r = 0.10; P = .49) or scores on the American Board of Pediatrics certification examinat ion (r = 0.22; P = .11). There were weak correlations between faculty ratin gs and scores of faculty interviews during the intern application process ( r = 0.27; P = .02) and scores on the pediatric in-service examination durin g PGY-1 (r = 0.28; P = .02). There was no difference in faculty ratings of residents who were elected to Alpha Omega Alpha during medical school (mean +/- SD, 3.32 +/- 1.21) as compared with those who were not (mean +/- SD, 3 .08 +/- 1.34) (P = .25). Conclusions: There is significant agreement among faculty raters about the clinical competence of pediatric residents. Medical school grades, performa nce on standardized examinations, interviews during the intern application process, and match-list ranking are not predictors of clinical performance during residency.