What is the shelf life of a test? The effect of time on the psychometrics of a cognitive ability test battery

The psychometric stability of the Armed Services Vocational Aptitude Batter y was studied with data collected at 5 points over a 16-year period using i tem response theory (IRT) methods. Although 25 of the 200 items changed sig nificantly over the years across 3 different gender-ethnic groups (i.e., Wh ite men, White women, and Black men), the overall characteristics of the te sts were not severely affected by item-level changes. Items from tests that were more semantically laden were found to be more susceptible to the effe cts of time compared with those that focused on skills and principles. The findings are discussed in the context of the effects of time on the effecti veness of psychological measures. A call is made to test developers and tes t users to pay attention to the shelf life of their tests. The use of IRT m ethods for studying the effects of time on psychometrics is also discussed.