COMPARISON BETWEEN EXACT AND PARAMETRIC DISTRIBUTIONS OF MULTIPLE INTER-RATERS AGREEMENT COEFFICIENT

The Global Kappa statistic can be used for measuring the degree of agr eement between more than two raters. In this paper, we give simple com putation formulas in the case of dichotomic and no missing responses i n order to describe the exact distribution of Kappa for various sets d efined by the number of raters, of different examinations and of posit ive responses. By programming those computations with the SAS(C) softw are package, we were able to compare the exact and parametric distribu tions of Kappa. Although the correlation between parameters is high, a linear regression analysis shows that a strong imbalance between posi tive and negative responses can lead to misinterpretation of the resul ts when using the parametric method.