VALIDITY OF ITEM SELECTION - A COMPARISON OF AUTOMATED COMPUTERIZED ADAPTIVE AND MANUAL PAPER AND PENCIL EXAMINATIONS

Background: As computerized adaptive testing (CAT) becomes more preval ent, it is important to confirm that the computerized adaptive and pap er-and-pencil (P&P) examinations offer comparable validity and statist ical performance. Purpose: The purpose of this study was to investigat e the validity and statistical properties of automated item selection (CAT) compared to manual item selection (P&P). Methods: A committee of specialists rated computerized adaptive tests (CATs) and P&P examinat ions with regard to face validity, adherence to test specifications, o rdering of items, and cognitive skill distribution. The psychometric p roperties were compared. Results: Results indicated that the CATs and P&P examinations were comparable for face, content, and construct vali dity, as well as psychometric characteristics. Conclusions: Test const ructed automatically by the computer or manually for P&P can meet the criteria for validity and statistical performance. These findings gene ralize to any carefully developed examination program.