Measurement equivalence between applicant and incumbent groups: An IRT analysis of personality scales

Incumbents are often used in the development and validation of a wide varie ty of personnel selection instruments, including noncognitive instruments s uch as personality tests. However, the degree to which assumed motivational factors impact the measurement equivalence and validity of tests developed using incumbents has not been adequately addressed. This study addressed t his issue by examining the measurement equivalence of 6 personality scales between a group applying for jobs as sales managers in a large retail organ ization (N = 999) and a group of sales managers currently employed in that organization (N = 796). A graded item response theory model (Samejima, 1969 ) was fit to the personality scales in each group. Results indicated that m oderately large differences existed in personality scale scores (approximat ely 1/2 standard deviation units) but only one of the six scales contained any items that evidenced differential item functioning and no scales eviden ced differential test functioning. Tn addition, person-level analyses showe d no apparent differences across groups in aberrant responding. The results suggest that personality measures used for selection retain similar psycho metric properties to those used in incumbent validation studies.