Judgmental item analysis of the Nedelsky and Angoff standard-setting methods

It was hypothesized that the Nedelsky versus the Angoff methods would have (a) lower intrajudge inconsistency and (b) lower cutscores, especially for items presenting a challenge to the judges. These hypotheses were tested an d supported in 3 standard-setting studies. These studies used 80 graduate s tudents in education as judges to set standards for exams of a research met hod course they were taking. Lower intrajudge inconsistency of the Nedelsky method is attributed to focusing on response options and making multiple d ecisions. The strengths of the Nedelsky method, however, are limited by its discrete judgmental estimates. It is suggested that combining the strong f eatures of both the Angoff and Nedelsky methods would make a stronger stand ard-setting procedure.