Defining error rates and power for detecting answer copying

Citation
Ja. Wollack et al., Defining error rates and power for detecting answer copying, APPL PSYC M, 25(4), 2001, pp. 385-404
Citations number
25
Categorie Soggetti
Psycology
Journal title
APPLIED PSYCHOLOGICAL MEASUREMENT
ISSN journal
01466216 → ACNP
Volume
25
Issue
4
Year of publication
2001
Pages
385 - 404
Database
ISI
SICI code
0146-6216(200112)25:4<385:DERAPF>2.0.ZU;2-T
Abstract
A familywise approach is described for evaluating the significance of copyi ng indices designed to hold the Type I error rate constant for each examine e. The empirical Type I error rate and power of two indices, omega (Wollack , 1997) and g(2) (Frary, Tideman, & Watts, 1977), are examined under a vari ety of copying situations. Results indicated that the traditional pairwise approach falsely detected examinees almost three times more often than the nominal alpha level. Familywise Type I error rates were substantially small er, although they also tended to be somewhat inflated at small alpha levels as the percentage of items copied increased. Eliminating the indices detec ting a source from the copier, in situations where the copier was also dete cted from the source, helped control the familywise Type I error rates for all alpha greater than or equal to .001. Lack of Type I error control meant power could not be evaluated for g(2) under any of the simulated familywis e conditions. Familywise power for omega was reasonable when at least 30% o f the items were copied.