Sl. Wise et al., Examinee judgments of changes in item difficulty: Implications for item review in computerized adaptive testing, APPL MEAS E, 12(2), 1999, pp. 185-198
We examined the degree to which providing item review on a computerized ada
ptive test (CAT) could be used by examinees to artificially inflate their s
cores. Kingsbury (1996) described a strategy in which examinees could use t
he changes in item difficulty during a CAT to determine which items' answer
s are incorrect and should be changed during item review. The results of ou
r first 2 studies suggest that examinees are not highly proficient at discr
iminating item difficulty-a skill needed for a successful application of th
e Kingsbury strategy. In the third study, we compared the Kingsbury strateg
y-which examinees would use only for guessed items-be a generalized strateg
y used for all sequential item pairs. The Kingsbury strategy yielded a smal
l average score gain, whereas the generalized strategy yielded an average s
core loss. These results suggest that only the Kingsbury strategy is likely
to enable examinees to successfully inflate their test scores.