ESTIMATING RATER AGREEMENT IN 2-X-2 TABLES - CORRECTION FOR CHANCE AND INTRACLASS CORRELATION

Many estimators of the measure of agreement between two dichotomous ra tings of a person have been proposed. The results of Fleiss (1975) are extended, and it is shown that four estimators-Scott's (1955) pi coef ficient, Cohen's (1960) (k) over cap, Maxwell & Pilliner's (1968) r(11 ,) and Mak's (1988) (p) over tilde-are interpretable both as chance-co rrected measures of agreement and as intraclass correlation coefficien ts for different ANOVA models. Relationships among these estimators ar e established for finite samples. Under Kraemer's (1979) model, it is shown that these estimators are equivalent in large samples, and that the equations for their large sample variances are equivalent.