SIMPLE MOMENT ESTIMATES OF THE KAPPA-COEFFICIENT AND ITS VARIANCE

Estimating equations are used to develop simple non-iterative estimate s of the kappa-coefficient that can be used when there are more than t wo random raters and/or unbalanced data (each subject is not judged by every rater). We show that there is a simple way to estimate the vari ance of any estimate of the kappa-coefficient that is a solution to an estimating equation. Two non-iterative estimates that are shown to be solutions to estimating equations are Fleiss's estimate and Schouten' s estimate. Also, assuming that the underlying data are beta-binomial, we compare the asymptotic relative efficiency of the non-iterative es timators Of kappa relative to the iterative maximum likelihood estimat or (MLE) of kappa from the beta-binomial distribution. Fleiss's estima tor was found to have high efficiency. Finally, simulations are used t o compare the finite sample performance of these estimators as well as the MLE from the beta-binomial distribution. In the simulations, the Newton-Raphson algorithm for the MLE from the beta-binomial model did not always converge in small samples, which also supports the use of a non-iterative estimate in small samples. The estimators are also comp ared by using a psychiatric data set given by Fleiss.