Evaluating the quality of systematic reviews in the emergency medicine literature

Study objective: The objective of this study was to examine the scientific quality of systematic reviews published in 5 leading emergency medicine jou rnals. Methods: MEDLINE and EMBASE databases were electronically searched to ident ify published systematic reviews, Searches were only conducted in emergency medicine journals during the past 10 years; 4 of the journals were also ha nd searched, Potential reviews were assessed independently by 2 reviewers f or inclusion. Data regarding methods were extracted from each review indepe ndently by 2 reviewers. All systematic reviews were retrieved and rated for quality by using the 10 questions from the overview quality assessment que stionnaire. Results: Twenty-nine reviews were identified from more than 100 citations. The overall scientific quality of the systematic reviews was low (mean scor e, 2.7; 95% confidence interval 2.1 to 3.2; maximum possible score, 7.0). S election and publication biases were rarely addressed in this collection of reviews. For example, the search strategies were only identified in 9 (31% ) reviews, whereas independent study selection (6 [21%]) and quality assess ment of included studies (9 [31%]) were infrequently performed. Overall, th e majority of reviews had extensive flaws, and only 3 (10%) had minimal fla ws. Conclusion: The results of the study indicate that many of the systematic r eviews published in the emergency medicine literature contain major flaws; reviews with poor methodology may limit the validity of reported results. F urther efforts should be made to improve the design, reporting, and publica tion of systematic reviews in emergency medicine.