Using rating augmentation to expand the scale of an analytic rubric

A method of expanding a rating scale 3-fold without the expense of defining additional benchmarks was studied. The authors used an analytic rubric rep resenting 4 domains of writing and composed of 4-point scales to score 120 writing samples from Georgia's 11th-grade Writing Assessment, The raters au gmented the scores of papers on which the proficiency levels appeared sligh tly higher or lower than the benchmark papers at the selected proficiency l evel by adding a "+" or a "-" to the score, The results of the study indica te that the use of this method of rating augmentation tends to improve most indices of interrater reliability, although the percentage of exact and ad jacent agreement decreases because of the increased number of rating possib ilities. In addition, there was evidence to suggest that the use of augment ation may produce domain-level scores with sufficient reliability for use w ith diagnostic feedback to teachers about the performance of students.