A PILOT-STUDY OF THE USE OF THE ANALYTIC HIERARCHY PROCESS FOR THE SELECTION OF SURGERY RESIDENTS

Purpose. To compare a specific decision-making process-the analytic hi erarchy process (AHP)-with the traditional informal selection process in the selection of general surgery residents. Method. The study focus ed on 1994 and 1995 applicants for the four positions in the five-year general surgery residency program at the Graduate Hospital in Philade lphia. Three criteria were used: academic performance, personal fit, a nd surgical appropriateness. The relative importance of each was deter mined by pairwise comparison. For each hierarchy level, these comparis ons were combined into a pairwise comparison matrix, and weights were deter mined for each criterion and rating category. The rating categor y weights for each criterion were scaled so that outstanding received the full criterion weight. Each applicant was interviewed by three com mittee members and rated with both the AHP system and the traditional 0-10 scoring system. In both cases the rating scores were averaged to create a single score for each applicant. The final ranking list (advo cacy ranking) was compiled at a meeting of the entire selection commit tee, during which each member spoke on behalf of the candidates he or she had interviewed. Results. Significant Spearman correlations were f ound between the AHP ranking and the traditional ranking in both years (1994: n = 26, r = .63, p = .0005; 1995: n = 25, r = .061, P = .0012) . The AHP ranking was also significantly correlated with the advocacy ranking in 1994 (n = 26, r = .43, P = .0273); however, there was no si gnificant correlation found in 1995. In 1994 the traditional ranking s ignificantly correlated with the advocacy ranking (n = 26, r = .40, p = .0423). This was not the case in 1995, suggesting that the results o f the interviewing process had minimal influence on the outcome of the selection process that year. Conclusion. The findings from this pilot study support the use of the AHP as a viable alternative for the sele ction of surgical residents. Although the small sample size limits the generalizability of the results, the AHP is a quantitative alternativ e to the traditional, unwieldy and subjective selection process. Quant itative assessment and ranking of all aspects of a candidate's attribu tes and performance allow a program to more closely match a candidate to that particular institution.