A retrospective evaluation of a data mining approach to aid finding new adverse drug reaction signals in the WHO International Database

Background: The detection of new drug safety signals is of growing importan ce with ever more new drugs becoming available and exposure to medicines in creasing. The task of evaluating information relating to safety lies with n ational agencies and, for international data, with the World Health Organiz ation Programme for International Drug Monitoring. Rationale: An established approach for identifying new drug safety signals from the international database of more than 2 million case reports depends upon clinical experts from around the world. With a very large amount of i nformation to evaluate, such an approach is open to human error. To aid the clinical review, we have developed a new signalling process using Bayesian logic, applied to data mining, within a confidence propagation neural netw ork (Bayesian Confidence Propagation Neural Network; BCPNN). Ultimately, th is will also allow the evaluation of complex variables. Methods: The first part of this study tested the predictive value of the BC PNN in new signal detection as compared with reference literature sources ( Martindale's Extra Pharmacopoeia in 1993 and July 2000, and the Physicians Desk Reference in July 2000). In the second part of the study, results with the BCPNN method were compared with those of the former signalling procedu re. Results: In the study period (the first quarter of 1993) 107 drug-adverse r eaction combinations were highlighted as new positive associations by the B CPNN, and referred to new drugs. 15 drug-adverse reaction combinations on n ew drugs became negative BCPNN associations in the study period. The BCPNN method detected signals with a positive predictive value of 44% and the neg ative predictive value was 85%. 17 as yet unconfirmed positive associations could not be dismissed with certainty as false positive signals. Of the 10 drug-adverse reaction signals produced by the former signal detec tion system from data sent out for review during the study period, 6 were a lso identified by the BCPNN. These 6 associations have all had a more than 10-fold increase of reports and 4 of them have been included in the referen ce sources. The remaining 4 signals that were not identified by the BCPNN h ad a small, or no, increase in the number of reports, and are not listed in the reference sources. Conclusion: Our evaluation showed that the BCPNN approach had a high and pr omising predictive value in identifying early signals of new adverse drug r eactions.