FEASIBILITY OF HOSPITAL-BASED USE OF PEER RATINGS TO EVALUATE THE PERFORMANCES OF PRACTICING PHYSICIANS

Purpose. To address the feasibility of obtaining reliable evaluations of individual physicians from peer ratings undertaken at diverse hospi tals. Method. Eleven hospitals in diverse locations in the United Stat es were recruited to participate. With the aid of the hospitals' medic al directors, up to 40 board-certified internists with admitting privi leges were recruited per hospital. Participating physicians provided d emographic data about themselves and nominated physician-associates to do peer ratings. Between April 1993 and January 1994, the physicians were rated by their peers, who received a single mailing with no follo w-up. The raters used a nine-point Likert scale for 11 cognitive and n oncognitive categories. Administrative procedures were coordinated fro m the American Board of Internal Medicine. Chi-square, Student's t-tes t, and factor analysis using varimax rotation were used to analyze the results. Results. Of the 4,139 questionnaires that were mailed to pee r raters, 3,005 (73%) were returned. Of the 228 physicians who were ra ted, 187 received ten or more usable ratings, which were used for furt her analysis. The findings confirmed the results of previous research: The highest mean rating was for the category of integrity, and the lo west was for the category of psychosocial aspects of care. Ten to 11 r esponses per physician were necessary to achieve a generalizability co efficient of .7. Nearly 90% of the variance in the ratings was account ed for by two factors, one representing cognitive and clinical managem ent skills and the other, humanistic qualities. For 16 physicians (9%) , the ratings of overall clinical skills were less than 7 on a scale f rom 1 (low) to 9 (high); their ratings for all individual cognitive an d noncognitive categories were below the ratings of the other physicia ns. Conclusion. The peer raters' response rate and the analysis of the ratings suggest that the rating process is acceptable to physicians a nd that it is feasible to obtain reliable, multidimensional peer evalu ations of individual physicians practicing in diverse clinical setting s.