Examinee judgments of changes in item difficulty: Implications for item review in computerized adaptive testing

We examined the degree to which providing item review on a computerized ada ptive test (CAT) could be used by examinees to artificially inflate their s cores. Kingsbury (1996) described a strategy in which examinees could use t he changes in item difficulty during a CAT to determine which items' answer s are incorrect and should be changed during item review. The results of ou r first 2 studies suggest that examinees are not highly proficient at discr iminating item difficulty-a skill needed for a successful application of th e Kingsbury strategy. In the third study, we compared the Kingsbury strateg y-which examinees would use only for guessed items-be a generalized strateg y used for all sequential item pairs. The Kingsbury strategy yielded a smal l average score gain, whereas the generalized strategy yielded an average s core loss. These results suggest that only the Kingsbury strategy is likely to enable examinees to successfully inflate their test scores.