ASSESSING THE DIMENSIONALITY OF ITEM RESPONSE MATRICES WITH SMALL SAMPLE SIZES AND SHORT TEST LENGTHS

The assumption of unidimensionality must be met to legitimately use co mmon item response theory models. The validity of score-based inferenc es rests largely on the extent to which it can be shown that the dimen sional structure underlying a test is consistent with the blueprint. L ittle research has been undertaken to examine the behavior of dimensio nality assessment procedures in conditions similar to those encountere d in small-volume administrations. The purpose of this study was to ex amine empirical Type I error rates and rejections rates for 3-dimensio nality assessment procedures with data sets simulated to reflect short tests and small samples. The TESTFACT (Wilson, Wood, gr Gibbons, 1991 ) G(2) difference test and the LISREL8 (Joreskog & Sorbom, 1993a) chi- square statistic suffered from an inflated Type I error rate with unid imensional data sets, whereas the approximate chi-square statistic bas ed on a NOHARM (Fraser Br McDonald, 1988) analysis did not. Rejection rates with simulated 2-dimensional data sets were high for air procedu res. The behavior of the G(2) difference test was highly influenced by the independent variables manipulated, which was not the case for the approximate chi-square statistic. The implications of these results f or small-volume administrations are discussed.