EVALUATING 2ND-ORDER PROBABILITY JUDGMENTS WITH STRICTLY PROPER SCORING RULES

Empirical studies have demonstrated that uncertainty about event proba bilities, also known as ambiguity or second-order uncertainty, can aff ect decision makers' choice preferences. Despite the importance of sec ond-order uncertainty in decision making, almost no effort has been di rected towards the development of methods that evaluate the accuracy o f second-order probabilities. In this paper, we describe conditions un der which strictly proper scoring rules can be used to assess the accu racy of second-order probability judgments. We investigate the effecti veness of using a particular strictly proper scoring rule - the ranked probability score - to discourage biased assessments of second-order uncertainty.