Identification of randomized controlled trials from the emergency medicineliterature: Comparison of hand searching versus MEDLINE searching

Study objective: As part of an ongoing project to identify all the randomiz ed controlled trials (RCTs) in the emergency medicine literature, in associ ation with the Cochrane Collaboration, 2 discrete studies were undertaken; the first, to compare motives for active participation in hand searching of the literature by emergency medicine professionals, and the second, to com pare hand searching with MEDLINE searching of a number of emergency medicin e journals. Methods: All listed members of the British Association for Emergency Medici ne (BAEM) and the Society for Academic Emergency Medicine (SAEM) received a standard letter outlining the objectives of the project, with 1 of 3 headi ngs assigned on an alternate basis. Recruited volunteers hand searched jour nals prioritized from the emergency medicine literature. Each issue of each journal was hand searched for RCTs. In addition, a comprehensive MEDLINE s earch was conducted for each journal. The yields of RCTs from the 2 searchi ng methods were compared for all journals and for each journal individually . Results: No clear motivation for participation in this work could be ascert ained because of the low response rates from BAEM and SAEM (10.1% and 1.8%, respectively). Only 18 (29.0%) of the 62 journals identified were indexed by MEDLINE. In the 14 journals indexed by MEDLINE for which hand searching was completed, a total of 710 RCTs were identified by a combination of the 2 approaches; of these, 592 (83.4%) were identified by hand searching alone and 483 (68.0%) by MEDLINE searching alone. Both methods identified 365 (5 1.4%) RCTs; hand searching revealed an additional 227 (32.0%) that were not identified by MEDLINE searching, and MEDLINE searching found 118(16.6%)tha t were not identified by hand searching. The difference between the proport ions identified by hand searching and by MEDLINE searching (15.4%; 95% conf idence interval [CI], 12.7% to 17.9%) was statistically significant (McNema r's chi(2) test, 1 df, 33.8; P<.0001). This difference was not significant for 8 of the journals. Conclusion: The response rates from mailing to members of the relevant prof essional organizations letters requesting participation in this work were v ery low and suggested that such an approach was not cost-effective. However , no formal costing exercise was undertaken. Searching results showed that, in the 14 emergency medicine journals indexed by MEDLINE for which hand se arching was completed, hand searching led to identification of additional R CTs (primary articles) not found through MEDLINE searching. However, hand s earching, although statistically significantly better than MEDLINE searchin g, failed to identify some of the RCTs found by MEDLINE searching, suggesti ng that hand searching is not a "gold standard" method and that the dual ap proach, promoted by the Cochrane Collaboration, may be the optimal approach for journals indexed by MEDLINE.