Assessing the comparability of school scores across test forms that are not parallel

Generalizability theory was used to assess the variation in school scores a cross ver short test forms that measured pupils' mathematics skills in Grad es 4 and 8. The forms had been linked, but the short forms were not paralle l in content or in their psychometric properties. In each grade, cutscores were set, and the analyses were conducted in terms of both scale scores and percentages at or above the cutscore. Large school-by-form interactions were found especially in Grade 8, where t he lack of parallelism impaired the equating between forms as well as the d ependability of school scores. The findings differed somewhat by school siz e and score metric. All results demonstrated both the lack of comparability in school scores across short, nonparallel test forms and the importance o f taking error into account when school scores are interpreted.