Management of scoring sessions in alternative assessment: the computer-assisted scoring approach

We describe a computer-assisted scoring approach in educational assessment. In this approach, scores are captured and analyzed as scoring takes place; information on scoring quality is used to provide immediate feedback to ra ters and make timely re-calibration and dismissal decisions. We present a c onceptual model for computer-assisted scoring and describe how we used this approach to manage the scoring sessions of an assessment for teacher certi fication. We found that computer-assisted scoring: (a) allowed us to provid e immediate feedback to raters about their scaring quality and make accurat e recalibration and dismissal decisions; and (b) did not affect the dependa bility of the scores or the flow of the scoring sessions. We also confirmed that, with appropriate software, raters can be trained readily to score co mplex performance with the aid of computers even when they have no prior ex perience with computers. The conceptual model allowed us to identify how cl ose our scoring sessions were to optimal efficiency. (C) 1999 Elsevier Scie nce Ltd. All rights reserved.