DOES MASKING AUTHOR IDENTITY IMPROVE PEER-REVIEW QUALITY - A RANDOMIZED CONTROLLED TRIAL

Context.-All authors may not be equal in the eyes of reviewers. Specif ically, well-known authors may receive less objective (poorer quality) reviews. One study at a single journal found a small improvement in r eview quality when reviewers were masked to author identity. Objective s.-To determine whether masking reviewers to author identity is genera lly associated with higher quality of review at biomedical journals, a nd to determine the success of routine masking techniques. Design and Setting.-A randomized controlled trial performed on external reviews o f manuscripts submitted to Annals of Emergency Medicine, Annals of Int ernal Medicine, JAMA, Obstetrics & Gynecology, and Ophthalmology. Inte rventions.-Two peers reviewed each manuscript. In one study arm, both peer reviewers received the manuscript according to usual masking prac tice. In the other arm, one reviewer was randomized to receive a manus cript with author identity masked, and the other reviewer received an unmasked manuscript. Main Outcome Measure.-Review quality on a 5-point Likert scale as judged by manuscript author and editor. A difference of 0.5 or greater was considered important. Results.-A total of 118 ma nuscripts were randomized, 26 to usual practice and 92 to intervention . In the intervention arm, editor quality assessment was complete for 77 (84%) of 92 manuscripts. Author quality assessment was complete on 40 (54%) of 74 manuscripts. Authors and editors perceived no significa nt difference in quality between masked (mean difference, 0.1; 95% con fidence interval [CI], -0.2 to 0.4) and unmasked (mean difference, -0. 1;95% CI, -0.5 to 0.4) reviews. We also found no difference in the deg ree to which the review influenced the editorial decision (mean differ ence, -0.1;95% CI,-0.3 to 0.3). Masking was often unsuccessful (overal l, 68% successfully masked; 95% CI, 58%-77%), although 1 journal had s ignificantly better masking success than others (90% successfully mask ed; 95% CI, 73%-98%). Manuscripts by generally known authors were less likely to be successfully masked (odds ratio, 0.3; 95% CI, 0.1-0.8). When analysis was restricted to manuscripts that were successfully mas ked, review quality as assessed by editors and authors still did not d iffer. Conclusions.-Masking reviewers to author identity as commonly p racticed does not improve quality of reviews. Since manuscripts of wel l-known authors are more difficult to mask, and those manuscripts may be more likely to benefit from masking, the inability to mask reviewer s to the identity of well-known authors may have contributed to the la ck of effect.