SIMULATION STUDIES OF THE EFFECTS OF SMALL SAMPLE-SIZE AND STUDIED ITEM PARAMETERS ON SIBTEST AND MANTEL-HAENSZEL TYPE-I ERROR PERFORMANCE

Two simulation studies investigated Type I error performance of two st atistical procedures for detecting differential item functioning (DIF) : SIBTEST and Mantel-Haenszel (MH). Because MH and SIBTEST are based o n asymptotic distributions requiring ''large'' numbers of examinees, t he first study examined Type I error for small sample sizes. No signif icant Type I error inflation occurred for either procedure. Because MH has the potential for Type I error inflation for non-Rasch models, th e second study used a markedly non-Rasch test and systematically varie d the shape and location of the studied item. When differences in dist ribution across examinee group of the measured ability were present, b oth procedures displayed inflated Type I error for certain items; MH d isplayed the greater inflation. Also, both procedures displayed statis tically biased estimation of the zero DIF for certain items, though SI BTEST displayed much less than MH. When no latent distributional diffe rences were present, both procedures performed satisfactorily under al l conditions.