What is the quality of the implemented meta-analytic procedures in chronicpain treatment meta-analyses?

Objective: Meta-analysis (MA) is the application of quantitative techniques for the purposes of summarizing data from individual studies. This type of review has many advantages over traditional reviews. However, different in vestigators performing MAs on the same data set have reached different conc lusions. These reliability problems have been attributed to differences in the quality of the implemented meta-analytic procedures. We, therefore, exa mined the chronic pain treatment meta-analytic literature for MA procedure quality and for the consistency of conclusions. Design, Setting, Participants, Outcome Measures: Chronic pain treatment MAs were isolated according to inclusion/exclusion criteria. Data from these M As were abstracted into structured tables. Table format reflected eight met a-analytic procedures identified previously as being important to MA implem entation quality. These were: adequacy of retrieval, publication bias, incl usion/exclusion criteria, abstraction of data, quality, homogeneity/heterog eneity, independence, and statistical versus clinical interpretation. Each meta-analytic procedure was then independently rated by two raters. Raring results were then analyzed by procedure for each individual MA for percenta ge scores out of 100%, and mean scores. For MAs addressing the same topic a rea (pain facility treatment, antidepressant treatment, manipulation treatm ent) direction of effect size was noted. Mean effect sizes were calculated for these subgroups. Results: Sixteen chronic pain treatment MAs fulfilled inclusion/exclusion c riteria. Mean procedure ratings indicated that four procedures may not be i mplemented adequately. These were publication bias, abstraction of data, qu ality, and homogeneity/heterogeneity. There was wide MA implementation scor e variability, with 37.5% scoring less than 50%. The effect sizes of the MA subgroups demonstrated replicate nonvariability. Conclusions: Some meta-analytic procedures could be interpreted to be imple mented inadequately in some chronic pain treatment MAs. There is wide varia bility between individual chronic pain treatment MAs on adequacy of impleme ntation of these procedures. However, the effect sizes of the different MA subgroups demonstrated consistency. This finding indicates that for these M A subgroups, MA results are consistent between authors. In addition, chroni c pain MAs, as compared with other groups of MAs, appear to address some of the procedures in a more adequate fashion. Future chronic pain MAs should concentrate on improving the quality of their methods with particular empha sis on the above four procedures. Because of potential validity problems wi th these results, these data cannot and should not be used to make administ rative decisions about previous MAs.