A SIMULATION STUDY OF METHODS FOR ASSESSING DIFFERENTIAL ITEM FUNCTIONING IN COMPUTERIZED ADAPTIVE TESTS

Simulated data were used to investigate the performance of modified ve rsions of the Mantel-Haenszel method of differential item functioning (DIF) analysis in computerized adaptive tests (CATs). Each simulated e xaminee received 25 items from a 75-item pool. A three-parameter logis tic item response theory (IRT) model was assumed, and examinees were m atched on expected true scores based on their CAT responses and estima ted item parameters. The CAT-based DIF statistics were found to be hig hly correlated with DIF statistics based on nonadaptive administration of all 75 pool items and with the true magnitudes of DIF in the simul ation. Average DIF statistics and average standard errors also were ex amined for items with various characteristics. Finally, a study was co nducted of the accuracy with which the modified Mantel-Haenszel proced ure could identify CAT items with substantial DIF using a classificati on system now implemented by some testing programs. These additional a nalyses provided further evidence that the CAT-based DIF procedures pe rformed well. More generally, the results supported the use of IRT-bas ed matching variables in DIF analysis.