CLINICAL-SIGNIFICANCE NOT STATISTICAL SIGNIFICANCE - A SIMPLE BAYESIAN ALTERNATIVE TO P VALUES

Objectives-To take the common ''Bayesian'' interpretation of conventio nal confidence intervals to its logical conclusion, and hence to deriv e a simple, intuitive way to interpret the results of public health an d clinical studies. Design and setting-The theoretical basis and pract icalities of the approach advocated is at first explained and then its use is illustrated by referring to the interpretation of a real histo rical cohort study. The study considered compared survival on haemodia lysis (HD) with that on continuous ambulatory peritoneal dialysis (CAP D) in 389 patients dialysed for end stage renal disease in Leicestersh ire between 1974 and 1985. Careful interpretation of the study was ess ential. This was because although it had relatively low statistical po wer, it represented all of the data that were available at the time an d it had to inform a critical clinical policy decision: whether or not to continue putting the majority of new patients onto CAPD. Measureme nts and analysis-Conventional confidence intervals are often interpret ed using subjective probability. For example, 95% confidence intervals are commonly understood to represent a range of values within which o ne may be 95% certain that the true value of whatever one is estimatin g really lies. Such an interpretation is fundamentally incorrect withi n the framework of conventional, frequency-based, statistics. However, it is valid as a statement of Bayesian posterior probability, provide d that the prior distribution that represents pre-existing beliefs is uniform, which means flat, on the scale of the main outcome variable. This means that there is a limited equivalence between conventional an d Bayesian statistics, which can be used to draw simple Bayesian style statistical inferences from a standard analysis. The advantage of suc h an approach is that it permits intuitive inferential statements to b e made that cannot be made within a conventional framework and this ca n help to ensure that logical decisions are taken on the basis of stud y results. In the particular practical example described, this approac h is applied in the context of an analysis based upon proportional haz ards (Cox) regression. Main results and conclusions-The approach propo sed expresses conclusions in a manner that is believed to be a helpful adjunct to more conventional inferential statements. It is of greates t value in those situations in which statistical significance may bear little relation to clinical significance and a conventional analysis using p values is liable to be misleading. Perhaps most importantly, t his includes circumstances in which an important public health or clin ical decision must be based upon a study that has unavoidably low stat istical power. However, it is also useful in situations in which a dec ision must be based upon a large study that indicates that an effect t hat is highly statistically significant seems too small to be of pract ical relevance. In the illustrative example described, the approach he lped in making a decision regarding the use of CAPD in Leicestershire during the latter half of the 1980s.