PUBLICATION BIAS - DIRECTION OF OUTCOME LESS IMPORTANT THAN SCIENTIFIC QUALITY

Publication bias, the tendency to publish work on the strength and dir ection of the results, is well recognised and its effect is to distort the body of scientific knowledge in favour of positive results. It ma y be especially important for complementary medicine (CM) which has a preponderance of positive reports in its specialist journals but which is also under-represented in the peer reviewed mainstream literature, To test whether orthodox scientific reviewers are biased against posi tive findings in CM papers, we selected 200 MEDLINE authors and random ly assigned them to four groups to receive a different version of the same fictitious short communication: a good (G) or bad (B) quality ver sion, each reporting either a positive ((+)) or negative ((-)) result. The B+ version had a significantly (P < 0.05) higher rejection score (mean 5.2, 95% CI 3.7 to 6.8) than the G(+) version (3.1, 1.8 to 4.4), but there were no significant differences between any other versions. Significantly more B+ were rejected than G(+) (55% vs 16%, chi(2) P < 0.05), The final acceptance score was closely related to the score fo r scientific quality (r = 0.82; P < 0.0001). There was therefore no ev idence of publication bias on the part of scientific reviewers as rega rds direction of study outcome but scientific quality of manuscript wa s found to be more important in terms of the accept/reject decision, a n encouraging result for the refereeing process.