THE EFFECT OF CHANGING DISEASE RISK ON CLINICAL REASONING

Objective: To assess the ability of health care professionals to evalu ate the effect of clinical test results in different settings. Design: Subjects were presented with a series of generic clinical scenarios i n which information about the test performance and the pretest probabi lity of disease was varied. The subject estimates of posttest probabil ity were compared with those calculated on the basis of Bayes' theorem . Participants: Fifty health care professionals, including 31 physicia ns and 19 nonphysicians, associated with a university teaching hospita l. Measurements and main results: Under a variety of testing condition s, both the physicians and the nonphysicians inaccurately estimated th e posttest probability of disease. Based on a logarithmic transformati on, the error in probability estimation was divided into a portion rel ated to the pretest probability of disease and a portion related to th e test performance. Most of the error in posttest probability estimati on was associated with the incorrect use of pretest probabilities. The subjects consistently overestimated the posttest probability of disea se expected under Bayes' theorem, with increasing error associated wit h decreasing pretest probability. Physician estimates of posttest prob ability increased with increasing likelihood ratios for each scenario. Nonphysician estimates of posttest probabilities increased with incre asing likelihood ratios for a positive test, but the estimates associa ted with a negative test result were inconsistent. Conclusions: Physic ians and nonphysicians overestimate posttest probabilities with increa sing error associated with decreasing disease risk. Some nonphysicians may not fully understand the effect of test performance on risk estim ation, particularly in the setting of a negative test. Health care pro fessionals should receive training in the proper evaluation of test in formation, with particular emphasis on the influence of pretest diseas e risk on the posttest probability of disease.