PREDICTION AND CROSS-VALIDATION OF NEURAL NETWORKS VERSUS LOGISTIC-REGRESSION - USING HEPATIC DISORDERS AS AN EXAMPLE

The authors developed and cross-validated prediction models for newly diagnosed cases of liver disorders by using logistic regression and ne ural networks, Computerized files of health care encounters from the F allen Community Health Plan were used to identify 1,674 subjects who h ad had liver-related health services between July 1, 1992, and June 30 , 1993, A total of 219 subjects were confirmed by review of medical re cords as incident cases, The 1,674 subjects were randomly and evenly d ivided into training and test sets, The training set was used to deriv e prediction algorithms based solely on the automated data; the test s et was used for cross-validation. The area under the Receiver Operatin g Characteristic curve for a neural network model was significantly la rger than that for logistic regression in the training set (p = 0.04), However, the performance was statistically equivalent in the test set (p = 0.45), Despite its superior performance in the training set, the generalizability of the neural network model is limited. Logistic reg ression may therefore be preferred over neural network on the basis of its established advantages, More generalizable modeling techniques fo r neural networks may be necessary before they are practical for medic al research.