Testing surgical skills of obstetric and gynecologic residents in a bench laboratory setting: Validity and reliability

OBJECTIVE: Resident surgical skills are acquired mainly through observing a nd later performing procedures in the operating room. Evaluation of surgica l skills has traditionally been done through subjective faculty evaluation, a technique that has poor reliability and unknown validity. Our goal was t o develop specific surgical tasks, both laparoscopic and open abdominal, th at could be objectively and reliably evaluated in a bench laboratory settin g. STUDY DESIGN: The prospective development of a reliable and valid resident surgical skills test in a bench laboratory setting was our goal. A written test of surgical knowledge and 12 skills tests were administered to 36 resi dents. Laparoscopic bench tasks were simulated with the use of a box and ca mera with a video display. Six laparoscopic tasks were assessed, including placing pegs on a board, running the bowel simulation, and other tasks that involve hand-eye coordination and manual dexterity. Open abdominal skills simulated incision closure, suturing a vaginal cuff, knot tying, and using a tie on a passer. Residents were timed at each given station and were give n a rating score by 2 examiners. RESULTS: Knowledge scores showed a significant improvement by residency lev el. Assessment of construct validity (the ability to discriminate among res idency levels) demonstrated significant differences on the rating of overal l performance and individual tasks by level (determined by 1-way analysis o f variance). Interrater reliability (agreement between 2 raters) with the u se of intraclass correlation was 0.79 for the total score. The cost to admi nister the bench laboratory test was less than $50 and required 30 hours of faculty time. CONCLUSION: The results of this study suggest that surgical bench laborator y tasks can assess residents' surgical skills with good reliability and val idity on most tasks. Our previous study, which used an animal laboratory, w as expensive, and the bench laboratory model may provide an alternative mea ns to assess surgical skills.