ISSUES IN COMPARISONS BETWEEN METAANALYSES AND LARGE TRIALS

Context.-The extent of concordance between meta-analyses and large tri als on the same topic has been investigated with different protocols. Inconsistent conclusions created confusion regarding the validity of t hese major tools of clinical evidence. Objective.-To evaluate protocol s comparing meta-analyses and large trials in order to understand if a nd why they disagree on the concordance of these 2 clinical research m ethods, Design.-Systematic comparison of protocol designs, study selec tion, definitions of agreement, analysis methods, and reported discrep ancies between large trials and meta-analyses. Results.-More discrepan cies were claimed when large trials were selected from influential jou rnals (which may prefer trials disagreeing with prior evidence) than f rom already performed meta-analyses (which may target homogeneous tria ls) and when both primary and secondary (rather than only primary) end points were considered. Depending on how agreement was defined, kappa coefficients varied from 0.22 (low agreement) to 0.72 (excellent agre ement), The correlation of treatment effects between large trials and meta-analyses varied from -0.12 to 0.76, but was more similar (0.50-0. 76) when only primary end points were considered, When both the magnit ude and uncertainty of treatment effects were considered, large trials disagreed with meta-analyses 10% to 23% of the time, Discrepancies we re attributed to different disease risks, variable protocols, quality, and publication bias. Conclusions.-Comparisons of large trials with m eta-analyses may reach different conclusions depending on how trials a nd meta-analyses are selected and how end points and agreement are def ined. Scrutiny of these 2 major research methods can enhance our appre ciation of both for guiding medical practice.