STANDARDIZING PROCEDURES FOR CALCULATING RORSCHACH INTERRATER RELIABILITY - CONCEPTUAL AND EMPIRICAL FOUNDATIONS

Although the Rorschach test has demonstrated significant refinements i n reliability, validity, and statistical power as a result of the proc edural standardization and scoring innovations introduced by Exner's C omprehensive System, the issue of Rorschach interrater reliability rem ains unexplored. This article examines the psychometric foundations of Rorschach interrater reliability and applies notions from applied beh avioral analysis to the treatment of Rorschach data. We empirically co mpare 3 methods of quantifying interrater agreement, their accuracy in estimating interrater agreement, and efficiency in reducing error in Rorschach research. Results indicate that the magnitude of differences between methods of quantifying interrater agreement and the associate d reductions of error are significant. We propose a standard method fo r quantifying interrater agreement in Rorschach research.