Normative and descriptive analyses of Simpson's Paradox in decision making

Suppose that you are evaluating two delivery companies. Your investigation shows that Company B has a better on-time rate for small packages and also for large packages. Despite Company B's performance, however, Company A has a better overall ontime rate. This situation exemplifies Simpson's Paradox , in which the judged relationship between two variables (e,g,, company and performance) differs depending on whether that relationship is viewed with in subcategories of a third variable (e,g,, package size) or in the aggrega te. A normative analysis is presented arguing that the reasonableness of us ing the third variable depends upon the sample size as well as the separati on between and variability within categories. To test subjects' abilities t o behave appropriately in Simpson's Paradox situations, we examined respons es to variations in these factors in five studies. Results showed that subj ects had little or no sensitivity to differing stimulus set sizes. Also, su bjects were sensitive to relationship strength and the variability within a nd between groups, and in nonnormative ways. Subjects' judgment behavior is related to a broader perspective concerning selection among multiple avail able levels of analysis based on a consideration of argument strength. (C) 2000 Academic Press.