APPLICATION OF PSYCHOMETRIC STANDARDS TO SCORING SYSTEM EVALUATION - IS NEW NECESSARILY IMPROVED

Psychometric guidelines are suggested that expand the evaluation of sc oring systems beyond the most frequently (and often singly) considered criterion, that of interrater reliability. Principles of reliability and validity are elaborated and discussed with regard to scoring syste ms. Also addressed are assays for efficiency in applying a new system, the effects of rater proficiency on outcome scores, the comparison of costs and benefits of new scoring systems, and the comparison of a ne w scoring system to an existing system to demonstrate its advantage. A s a timely example to illustrate these principles, a new scoring syste m is introduced for the Rey-Osterrieth Complex Figure Test and is comp ared to the most popular scoring system for this test (Lezak, 1983, p. 400). The proposed principles would be relevant to any scoring system that codes subjective behavioral data into objective values.