Assessing interrater agreement on the job relevance of a test: A comparison of the CVI, T, (rWG(J)), and r*(WG(J)) indexes

This article examines the problem of estimating interrater agreement about the job relevance of test items by examining 4 different indexes: C. H. Law she's (1975) Content Validity Index (CVI); H. E. A. Tinsley and D. J. Weiss 's (1975) T; L. R. James, R. G. Demaree, and G. Wolf's (1984) r(WG(J)); and M. K. Lindell, C. J. Brandt, and D. J. Whitney's (1999) r*(WG(J)). These 4 indexes are shown to be systematically related to each other, but r*(WG(J) ) is computationally simpler and is more generally applicable than the othe r indexes.