EMPIRICAL-EVALUATION OF PRIOR BELIEFS ABOUT FREQUENCIES - METHODOLOGYAND A CASE-STUDY IN CONGENITAL HEART-DISEASE

We consider the problem of critiquing prior beliefs concerning the dis tribution of a discrete random variable in the light of a sequentially obtained sample. A topical application concerns a probabilistic exper t system for the diagnosis of congenital heart disease, which requires specification of a large number of conditional probabilities that are initially imprecisely estimated by a suitable ''expert.'' These prior beliefs may be formally updated as data become available, but it woul d seem essential to contrast the original expert assessments with the data obtained to quickly identify inappropriate subjective inputs. We consider both Bayes factor and significance testing techniques for suc h a prior/data comparison, both in nonsequential and sequential forms. The common basis as alternative standardizations of the logarithm of the predictive ordinate of the observed data is emphasised, and a Baye sian discrepancy statistic with a variety of interpretations provides a formal means of discounting the expert's judgments in the light of t he data. The judgments are found to be of generally high quality, and procedures for automatic monitoring and adaptation are recommended.