The Mt. Hood challenge: cross-testing two diabetes simulation models

Starting from identical patients with type 2 diabetes, we compared the 20-y ear predictions of two computer simulation models, a 1998 version of the IM IB model and version 2.17 of the Global Diabetes Model (GDM). Primary measu res of outcome were 20-year cumulative rates of: survival, first (incident) acute myocardial infarction (AMI), first stroke, proliferative diabetic re tinopathy (PDR), macro-albuminuria (gross proteinuria, or GPR), and amputat ion. Standardized test patients were newly diagnosed males aged 45 or 75, w ith high and low levels of glycated hemoglobin (HbA,,), systolic blood pres sure (SBP), and serum lipids. Both models generated realistic results and a ppropriate responses to changes in risk factors. Compared with the GDM, the IMIB model predicted much higher rates of mortality and AMI, and fewer str okes. These differences can be explained by differences in model architectu re (Markov vs, microsimulation), different evidence bases for cardiovascula r prediction (Framingham Heart Study cohort vs. Kaiser Permanente patients) , and isolated Versus interdependent prediction of cardiovascular events. C ompared with IMIB, GDM predicted much higher lifetime costs, because of low er mortality and the use of a different costing method. It is feasible to c ross-validate and explicate dissimilar diabetes simulation models using sta ndardized patients. The wide differences in the model results that we obser ved demonstrate the need for cross-validation. We propose to hold a second 'Mt Hood Challenge' in 2001 and invite all diabetes modelers to attend. (C) 2000 Elsevier Science Ireland Ltd. All rights reserved.