EVALUATING AN AUTOMATICALLY SCORABLE, OPEN-ENDED RESPONSE TYPE FOR MEASURING MATHEMATICAL REASONING IN COMPUTER-ADAPTIVE TESTS

The first generation of computer-based tests depends largely on multip le-choice items and constructed-response questions that can be scored through literal matches with a key. This study evaluated scoring accur acy and item functioning for an open-ended response type where correct answers, posed as mathematical expressions, can take many different s urface forms. Items were administered to 1,864 participants in field t rials of a new admissions test for quantitatively oriented graduate pr ograms. Results showed automatic scoring to approximate the accuracy o f multiple-choice scanning, with all processing errors stemming from e xaminees improperly entering responses. In addition, the items functio ned similarly in difficulty, item-total relations, and male-female per formance differences to other response types bring considered for the measure.