P. Narayanan et H. Swaminathan, PERFORMANCE OF THE MANTEL-HAENSZEL AND SIMULTANEOUS ITEM BIAS PROCEDURES FOR DETECTING DIFFERENTIAL ITEM FUNCTIONING, Applied psychological measurement, 18(4), 1994, pp. 315-328
Two nonparametric procedures for detecting differential item functioni
ng (DIF)-the Mantel-Haenszel (MH) procedure and the simultaneous item
bias (SIB) procedure-were compared with respect to their Type I error
rates and power. Data were simulated to reflect conditions varying in
sample size, ability distribution differences between the focal and re
ference groups, proportion of DIF items in the test, DIF effect sizes,
and type of item. 1,296 conditions were studied. The SIB and MH proce
dures were equally powerful in detecting uniform DIF for equal ability
distributions. The SIB procedure was more powerful than the MH proced
ure in detecting DIF for unequal ability distributions. Both procedure
s had sufficient power to detect DIF for a sample size of 300 in each
group. Ability distribution did not have a significant effect on the S
IB procedure but did affect the MH procedure. This is important becaus
e ability distribution differences between two groups often are found
in practice. The Type I error rates for the MH statistic were well wit
hin the nominal limits, whereas they were slightly higher than expecte
d for the SIB statistic. Comparisons between the detection rates of th
e two procedures were made with respect to the various factors.