THIN VERSUS THICK MATCHING IN THE MANTEL-HAENSZEL PROCEDURE FOR DETECTING DIF

This Monte Carlo study examined strategies for forming the matching va riable for the Mantel-Haenszel (MH) differential item functioning (DIF ) procedure; thin matching on total test score was compared to forms o f thick matching, pooling levels of the matching variable. Data were g enerated using a three-parameter logistic (3PL) item response theory ( IRT) model with common guessing parameter. Number of subjects and test length were manipulated, as were the difficulty, discrimination, and presence/absence of DIF in the studied item. Outcome measures were the transformed log-odds DELTA(MH), its standard error, and the MH chi-sq uare statistic. For short tests (5 or 10 items), thin matching yielded very poor results, with a tendency to falsely identify items as posse ssing DIF against the reference group. The best methods of thick match ing yielded outcome measure values closer to the expected value for no n-DIF items, as well as a larger value than thin matching when the stu died item possessed DIF. Intermediate length tests yielded similar res ults for thin matching and the best methods of thick matching. The met hod of thick matching that performed best depended on the measure used to detect DIF. Both difficulty and discrimination of the studied item were found to have a strong effect on the value of DELTA(MH).