SHOULD UNPUBLISHED DATA BE INCLUDED IN METAANALYSES - CURRENT CONVICTIONS AND CONTROVERSIES

Objective.-To identify the extent to which meta-analyses currently inc lude unpublished data and whether editors, meta-analysts, and methodol ogists believe unpublished material should be included. Design.-This a rticle describes two related studies: a literature review and a cross- sectional survey. Sample Selection.-For the literature review, we iden tified all articles indexed by the key word meta-analysis from January 1989 to February 1991 and determined whether unpublished material had been searched for, obtained, and included in the meta-analyses. For t he cross-sectional survey, we surveyed authors of these meta-analyses, authors of articles addressing methodological issues in meta-analysis published during the same period, and editors of journals in which bo th types of articles were published. Intervention.-We asked the respon dents about their attitudes concerning inclusion of unpublished data a nd publication of articles from which data had previously been include d in a scientific overview. Main Outcome Measures.-Inclusion of unpubl ished data and opinions about whether unpublished material should be i ncluded in overviews and whether prior inclusion of data in an overvie w should bear on publication. Results.-Of 150 meta-analyses, 46 (30.7% ) included unpublished data in their primary analysis. Of authors surv eyed, 85% responded. Of the meta-analysts and methodologists, 77.7% fe lt that unpublished material should definitely or probably be included in scientific overviews; this was true of 46.9% of the editors. A tot al of 86.4% of the meta-analysts and methodologists and 53.2% of the e ditors felt that inclusion of data in a prior overview should have no bearing on full publication of original research. Conclusion.-While in clusion of unpublished data in scientific overviews remains controvers ial, most investigators directly involved in meta-analysis believe tha t unpublished data should not be systematically excluded. The most val id synthesis of available information will result when meta-analysts s ubject published and unpublished material to the same rigorous methodo logical evaluation and present results with and without unpublished so urces of data.