VALIDATING INFERENCES FROM NATIONAL ASSESSMENT OF EDUCATIONAL-PROGRESS ACHIEVEMENT-LEVEL REPORTING

The validity of interpretations of National Assessment of Educational Progress (NAEP) achievement levels is evaluated by focusing on evidenc e regarding 3 types of discrepancies: (a) discrepancies between standa rds implied by judgments of different types of items (e.g., multiple c hoice vs. short answer or dichotomously scored vs. extended response t asks scored using multipoint rubrics), (b) discrepancies between descr iptions of achievement levels with their associated exemplar items and the location of cut scores on the scale, and (c) discrepancies betwee n the assessments and content standards. Large discrepancies of all 3 types raise serious questions about some of the more expansive inferen ces that have been made in reporting NAEP results in terms of achievem ent levels. It is argued that the evidence reviewed provides a strong case for making more modest inferences and interpretations of achievem ent levels than have frequently been made.