D. Budescu et M. Barhillel, TO GUESS OR NOT TO GUESS - A DECISION-THEORETIC VIEW OF FORMULA SCORING, Journal of educational measurement, 30(4), 1993, pp. 277-291
Multiple-choice tests are often scored by formulas under which the res
pondent's expected score for an item is the same whether he or she omi
ts it or guesses at random. Typically, these formulas are accompanied
by instructions that discourage guessing. In this article, we look at
test taking from the normative and descriptive perspectives of judgmen
t and decision theory. We show that for a rational test taker, whose g
oal is the maximization of expected score, answering is either superio
r or equivalent to omitting-a fact which follows from the scoring form
ula. For test takers who are not fully rational, or have goals other t
han the maximization of expected score, it is very hard to give adequa
te formula scoring instructions, and even the recommendation to answer
under partial knowledge is problematic (though generally beneficial).
Our analysis derives from a critical look at standard assumptions abo
ut the epistemic states, response strategies, and strategic motivation
s of test takers. In conclusion, we endorse the number-right scoring r
ule, which discourages omissions and is robust against variability in
respondent motivations, limitations in judgments of uncertainty, and i
tem vagaries.