A HIERARCHICAL APPROACH TO INFERENCES CONCERNING INTEROBSERVER AGREEMENT FOR MULTINOMIAL DATA

We consider inference methods for interobserver agreement studies char acterized by two raters and several outcome categories that one can na turally combine to address a series of questions of a priori interest. We propose a new method based on a series of nested, statistically in dependent inferences, each corresponding to a binary outcome variable obtained by combining a substantively relevant subset of the original categories. We conduct the inferences using a goodness-of-fit procedur e that extends the approach of Donner and Eliasziw. The methodology pr esented is an alternative to methodology that places each of the outco me categories on an equal footing in estimating interobserver agreemen t for multinomial data. We provide two examples. (C) 1997 by John Wile y & Sons, Ltd.