RELIABILITY OF CREDENTIALING EXAMINATIONS AND THE IMPACT OF SCORING MODELS AND STANDARD-SETTING POLICIES

This article provides a brief history of developments in the assessmen t of reliability of credentialing examinations and presents some new r esults that highlight the complex interactions among examination scori ng, standard setting, and the reliability and validity of pass-fail de cisions. With respect to the second purpose, these interactions have b ecome especially important recently because of the use of performance assessments in credentialing examinations. The main points of the arti cle are that decision consistency is an important concept in evaluatin g credentialing examinations and that an estimate of the error in scor es near the passing score or standard is essential. Also, the influenc e of scoring models and standard-setting policies on decision consiste ncy and accuracy can be substantial. These factors need to be consider ed in arriving at final decisions about examination scoring and standa rd setting.