THE EFFECT OF DATA SAMPLING ON THE PERFORMANCE EVALUATION OF ARTIFICIAL NEURAL NETWORKS IN MEDICAL DIAGNOSIS

Purpose. To study the effect of data sampling on the predictive assess ment of artificial neural networks (ANNs) for medical diagnostic tasks . Methods. Three statistical techniques were used to evaluate the diag nostic performances of ANNs: 1) cross validation, 2) round robin, and 3) bootstrap. These techniques are different sampling plans designed t o reduce the small-sample estimation bias and variance contributions. The study was based on two networks, one developed for the diagnosis o f pulmonary embolism (1,064 cases) and the other developed for the dia gnosis of breast cancer (206 cases). Results. The three sampling techn iques produced different performance estimates for both networks. The estimates varied substantially depending on the training sample size a nd the training-stopping criterion. Conclusion. The predictive assessm ent of ANNs in medical diagnosis can vary substantially based on the c omplexity of the problem, the data sampling technique, and the number of cases available.