EFFECT OF CHANGING PATIENT MIX ON THE PERFORMANCE OF AN INTENSIVE-CARE UNIT SEVERITY-OF-ILLNESS MODEL - HOW TO DISTINGUISH A GENERAL FROM ASPECIALTY INTENSIVE-CARE UNIT

Objective: To analyze the effects of patient mix diversity on performa nce of an intensive care unit (ICU) severity-of-illness model, Design: Multiple patient populations were created using computer simulations, A customized version of the Mortality Probability Model (MPM) II admi ssion model was used to ascertain probabilities of hospital mortality, Performance of the model was assessed using discrimination (area unde r the receiver operating characteristic curve) and calibration (goodne ss-of-fit testing), Setting: Intensive care units. Patients: Data were collected from 4,224 ICU patients from two Massachusetts hospitals (B aystate Medical Center, Springfield, MA; University of Massachusetts M edical Center, Worcester, MA) and two New York hospitals (Albany Medic al Center, Albany, NY; Ellis Hospital, Schenectady, NY), Interventions : Random samples were taken from a database, The percentage of patient s with each model disease characteristic was varied by assigning weigh ts (ranging from 0 to 10) to patients with a disease characteristic. T hree simulations were run for each of 15 model variables at each of 16 weights, totaling 720 simulations. Measurements and Main Results: The area under the receiver operating characteristic curve and model fit were assessed in each random sample. Removing patients with a given di sease characteristic did not affect discrimination or calibration. Inc reasing frequency of patients with each disease characteristic above t he original frequency caused discrimination and calibration to deterio rate. Model fit was more robust to increases in less frequently occurr ing patient conditions, From the goodness-of-fit test, a critical perc entage for each admission model variable was determined for each disea se characteristic, defined as the percentage at which the average p va lue for the test over the three replications decreased to < .10. Concl usions: The concept of critical percentages is potentially clinically important. It might provide an easy first step in checking applicabili ty of a given severity of illness model and in defining a general medi cal surgical ICU, If the critical percentages are exceeded, as might o ccur in a highly specialized ICU, the model would not be accurate, Alt ernative modeling approaches might be to customize the model coefficie nts to the population for more accurate probabilities or to develop sp ecialized models, The MPM approach remained robust for a large variati on in patient mix factors.