A psychometric experiment in causal inference to estimate evidential weights used by epidemiologists

A psychometric experiment in causal inference was performed on 159 Australi an and New Zealand epidemiologists. Subjects each decided whether to attrib ute causality to 12 summaries of evidence concerning a disease and a chemic al exposure. The 1,748 unique summaries embodied predetermined distribution s of 19 characteristics generated by computerized evidence simulation. Effe cts of characteristics of evidence on causal attribution were estimated fro m logistic regression, and interactions were identified from a regression t ree analysis. Factors with the strongest influence on the odds of causal at tribution were statistical significance (odds ratio = 4.5 if 0.001 less tha n or equal to P < 0.05 and 7,2 if P < 0.001, us P greater than or equal to 0.05); refutation of alternative explanations (odds ratio = 8.1 for no know n confounder vs none adjusted); strength of association (odds ratio = 2.0 i f 1.5 < relative risk <less than or equal to> 2.0 and 3.6 if relative risk > 2.0, vs relative risk less than or equal to 1.5); and adjunct information concerning biological, factual, and theoretical coherence. The refutation of confounding reduced the cutpoint in the regression tree for decision-mak ing based on strength of association. The effect of the number of supportiv e studies reached saturation after it exceeded 12 studies. There was eviden ce of flawed logic in the responses concerning specificity of effects of ex posure and a tendency to discount evidence if the P-value was a "near miss" (0.050 < P < 0.065). Evidential weights based on regression coefficients f or causal criteria can be applied to actual scientific evidence.