APPLYING THE MANTEL-HAENSZEL PROCEDURE TO COMPLEX SAMPLES OF ITEMS

This Monte Carlo study examined the effect of complex sampling of item s on the measurement of differential item functioning (DIF) using the Mantel-Haenszel procedure. Data were generated using a 3-parameter log istic item response theory model according to the balanced incomplete block (BIB) design used in the National Assessment of Educational Prog ress (NAEP). The length of each block of items and the number of DIF i tems in the matching variable were varied, as was the difficulty, disc rimination, and presence of DIF in the studied item. Block booklet, po oled booklet, and extra-information analyses were compared to a comple te data analysis using the transformed log-odds on the delta scale. Th e pooled booklet approach is recommended for use when items are select ed for examinees according to a BIB design. This study has implication s for DIF analyses of other complex samples of items, such as computer administered testing or another complex assessment design.