COMPARING CORRELATED KAPPAS BY RESAMPLING - IS ONE LEVEL OF AGREEMENTSIGNIFICANTLY DIFFERENT FROM ANOTHER

Researchers comparing diagnostic systems or screening tests frequently need to compare indices of agreement such as the kappa coefficient. W hile asymptotic methods exist for comparing kappas derived from indepe ndent samples, no satisfactory approach exists for comparing kappas de rived from the same or related samples. Resampling methods for compari ng kappa coefficients obtained from the same sample are presented. Eas ily undertaken using readily available software, these methods are ill ustrated by application to a small sample of psychiatric data, as well as to several thousand samples of simulated data. An acceptable type I error rate was exhibited. Resampling techniques - easily implemented and making few assumptions - deserve wider application in psychiatric research. Copyright (C) 1996 Elsevier Science Ltd.