EMPIRICAL-EVIDENCE OF BIAS - DIMENSIONS OF METHODOLOGICAL QUALITY ASSOCIATED WITH ESTIMATES OF TREATMENT EFFECTS IN CONTROLLED TRIALS

Objective.-To determine if inadequate approaches to randomized control led trial design and execution are associated with evidence of bias in estimating treatment effects. Design.-An observational study in which we assessed the methodological quality of 250 controlled trials from 33 meta-analyses and then analyzed, using multiple logistic regression models, the associations between those assessments and estimated trea tment effects. Data Sources.-Meta-analyses from the Cochrane Pregnancy and Childbirth Database. Main Outcome Measures.-The associations betw een estimates of treatment effects and inadequate allocation concealme nt, exclusions after randomization, and lack of double-blinding. Resul ts.-Compared with trials in which authors reported adequately conceale d treatment allocation, trials in which concealment was either inadequ ate or unclear (did not report or incompletely reported a concealment approach) yielded larger estimates of treatment effects (P<.001). Odds ratios were exaggerated by 41% for inadequately concealed trials and by 30% for unclearly concealed trials (adjusted for other aspects of q uality). Trials in which participants had been excluded after randomiz ation did not yield larger estimates of effects, but that lack of asso ciation may be due to incomplete reporting. Trials that were not doubl e-blind also yielded larger estimates of effects (P=.01), with odds ra tios being exaggerated by 17%. Conclusions.-This study provides empiri cal evidence that inadequate methodological approaches in controlled t rials, particularly those representing poor allocation concealment, ar e associated with bias. Readers of trial reports should be wary of the se pitfalls, and investigators must improve their design, execution, a nd reporting of trials.