Validation of logistic regression models in small samples: Application to calvarial lesions diagnosis

We have used the leave one out (LOO) method and the area under the receiver operating characteristic (ROC) curve to validate logistic models with a sa mple of 167 patients with calvarial lesions. Seven logistic regression mode ls were developed from 12 clinical and radiological variables to predict th e most common diagnoses separately. The LOO method was used to test the val idity of the equations. The discriminant power of every model was assessed by means of the area under the ROC curve (A(z)). The model with the greates t discrimination ability for the whole data set was the osteoma equation (A (z) = 0.951). The discriminatory ability of the statistical models decrease d significantly with the LOO procedure, having the malignancy model the hig hest value (A(z) = 0.931). The LOO method can obtain a high benefit from sm all samples in order to validate prediction rules. In studies with small sa mples, resampling techniques such as the LOO should be routinely used in pr edictive modeling. This method may improve the forecast of infrequent disea ses, such as calvarial lesions. J CLIN EPIDEMIOL 52;3:237-241, 1999. (C) 19 99 Elsevier Science Inc.