Measuring operative performance after laparoscopic skills training: Editedvideotape versus direct observation

Background and Purpose: Global assessment by direct observation has been va lidated for evaluating operative performance of surgery residents after for mal skills training but is time-consuming. The purpose of this study was to compare global assessment performed from edited videotape with scores from direct observation. Materials and Methods: Junior surgery residents (N = 22) were randomized to 2 weeks of formal videotrainer skills training or a control group. Laparos copic cholecystectomy was performed at the beginning and end of the rotatio n, and global assessment scores were compared for the training and control groups. Laparoscopic videotapes were edited: initial (2 minutes), cystic du ct/artery (6 minutes), and fossa dissection (2 minutes). Two independent ra ters performed both direct observation and videotape assessments, and score s were compared for each rater and for interrater reliability using a Spear man correlation. Results: Correlation coefficients for videotape versus direct observation f or five global assessment criteria were <0.33 for both raters (NS for all v alues). The correlation coefficient for interrater reliability for the over all score was 0.57 (P = 0.01) for direct observation v 0.28 (NS) for videot ape. The trained group had significantly better overall performance than th e control group according to the assessment by direct observation (P = 0.02 ) but not by videotape assessment (NS). Conclusions: Direct observation demonstrated improved overall performance o f junior residents after formal skills training on a videotrainer. Global a ssessment from an edited 10-minute videotape did not correlate with direct observation and had poor interrater reliability. Efficient and valid method s of evaluating operative performance await development.