SIGNAL VERSUS NOISE IN THE EVIDENCE BASE FOR MEDICINE - AN ALTERNATIVE TO HIERARCHIES OF EVIDENCE

Clinical practice frequently generates questions that are not easily a nswered by randomized trials. On conventional hierarchies of evidence, 'weaker' study designs are often more feasible. Also, much research i s not well designed. Yet we still need to make best use of the availab le evidence. Systematic reviews must therefore address the danger of u nderestimating the evidence from relevant literature if it includes on ly that of a certain methodological quality. This would run the risk o f missing or distorting the true message that the review is trying to identify. We propose a classification of research which does not rejec t studies on the basis of design alone, but recognizes the importance of assessing the message or 'signal' within each piece of research. It explicitly introduces judgement into the appraisal and synthesis of e vidence, and affords more flexibility in attaching weight to evidence that might otherwise be lost. It includes an assessment of methodologi cal quality, balancing this against the weight of its message, rather than rejecting studies which are below a certain threshold for quality . Fundamentally flawed research will still be rejected, but more commo nly papers can still be used, tempering the importance that we attach to their signal by the amount of 'noise' around that signal. The balan ce of these two elements may be termed the 'signal to noise ratio'.