FURTHER INVESTIGATION OF NONFUNCTIONING OPTIONS IN MULTIPLE-CHOICE TEST ITEMS

This study addressed the hypothesis that, after the systematic elimina tion of nonfunctioning options, four-option test items would perform a s well as five-option test items having one or more dysfunctional dist racters. The study consisted of two investigations involving an examin ation administered to 700 candidates for certification in a medical sp ecialty. In the first investigation, it was found that content experts exhibited a high degree of accuracy in identifying nonfunctioning opt ions where the criterion was empirical item analysis data. The second phase of the study compared five-option versions of multiple-choice it ems with four-option versions in which a nonfunctioning option had bee n removed. Results indicated that (a) removal of a nonfunctioning opti on resulted in a slight, nonsignificant overall increase in item diffi culty and no significant differences in item discrimination, (b) a tes t consisting of items with a nonfunctioning option removed was nearly equally reliable compared with a set of the same items in a five-optio n format, and (c) the use of empirical or judgmental methods of identi fying nonfunctioning options was not related to changes in item perfor mance. Implications, cautions, and suggestions for future research are provided.