An empirical comparison of three- and four-choice items and tests: Susceptibility to testwiseness and internal consistency reliability

Theoretical and test simulation work reveals that under the knowledge-or-ra ndom-guessing assumption, three-option item tests are at least as good as f our-option item tests in terms of item discrimination and internal consiste ncy. Of concern, however, is the finding that multiple-choice items may be susceptible to testwiseness, thereby contradicting the random-guessing assu mption. Both item-level and test-level characteristics were examined for it ems included in a high stakes school-leaving mathematics examination. As ex pected, the influence of testwiseness is lessened when three-option items a re used instead of four-option items. Differences and nondifferences betwee n the psychometric characteristics of the three-option and four-option test forms tend to agree with the findings of earlier studies: Tests consisting of three-option items are at least equivalent to tests composed of four op tions in terms of internal consistency score reliability, difficulty is inv ersely related to the number of options, and the findings for item discrimi nation are not conclusive.