The effect of instructions on multiple-choice test scores

Most standardized tests instruct subjects to guess under scoring procedures that do not correct for guessing or correct only for expected random guess ing. Other scoring rules, such as offering a small reward for omissions or punishing errors by discounting more than expected from random guessing, ha ve been proposed. This study was designed to test the effects of these four instruction/scoring conditions on performance indicators and on score reli ability of multiple-choice tests. Some 240 participants were randomly assig ned to four conditions differing in how much they discourage guessing. Subj ects performed two psychometric computerized tests, which differed only in the instructions provided and the associated scoring procedure. For both te sts, our hypotheses predicted (0) an increasing trend in omissions (showing that instructions were effective); (1) decreasing trends in wrong and righ t responses; (2) an increase in reliability estimates of both number right and scores. Predictions regarding performance indicators were mostly fulfil led, but expected differences in reliability failed to appear. The discussi on of results takes into account not only psychometric issues related to gu essing, but also the misleading educational implications of recommendations to guess in testing contexts.