Field-by-field evaluation of intraoperative transoesophageal echocardiography interpretative skills

A quality assurance system is essential for the credibility and structured growth of anaesthesiology-based transoesophageal echocardiography (TEE) pro grammes. We have developed software (Q/A Kappa), involving a 400-line sourc e code, capable of directly reporting kappa correlation coefficient values, using external reviewer interpretations as the 'gold standard', and thereb y allowing systematic assessment of the validity of intraoperative echocard iographic interpretation. This paper presents assessment of the validity of 240 intraoperative anaesthesiologists' echocardiographic interpretations, and, in addition, the results of field testing of this prototypical softwar e. Data, derived from consecutive cardiac surgery patients, consisted of stand ardized two-dimensional transoesophageal echocardiographic, colour flow and Doppler imaging sequences. Intraoperative and off-line 'gold standard' TEE interpretations were compared for 19 fields or variables using the Q/A Kap pa program. The kappa correlation coefficients were highly variable and dependent on th e examination field, ranging from 0.08 for apical regional wall motion scor es to 1.00 for tricuspid regurgitation grade, left atrial measurement, aort ic valve anatomy and left ventricular long axis and short axis global funct ion. The correlation coefficients were also operator dependent. These data (480 interpretations) were also manually integrated into the equation requi red for calculation of values of the variable kappa correlation coefficient , The relationship between Q/A Kappa-derived values and manually calculated values was highly significant (p < 0.001; r = 1.0). The implications and possible explanations of the results for particular ex amination fields are discussed. This study also demonstrates successful sea mless functioning of this software program from data entry, segmentation in to tables and valid statistical analysis, These findings suggest that it is practical to provide sophisticated continuous quality improvement TEE data on a routine basis.