IMPROVING THE QUALITY OF THE NCQA ANNUAL MEMBER HEALTH-CARE SURVEY VERSION 1.0

The National Committee for Quality Assurance (NCQA) developed a standa rdized survey instrument in 1995 designed to measure enrollee satisfac tion with the care and services received from health plans across the United States. After the survey was administered for a large number of health plans and thousands of responses were received, some areas for survey improvement have emerged. The objective of this research was t o evaluate the NCQA Annual Member Health Care Survey Version 1.0 (the standard form) relative to an alternate survey form created in coopera tion with The HMO Group, Maritz Marketing Research, Inc., and Healthca re Research Systems, Ltd. The alternate form of the NCQA survey was co nstructed to test several theories of measurement improvement via rewo rding of items, reordering of items, deletion of items, and addition o f items. The most important findings of the research project are repor ted herein. Ten geographically dispersed member health maintenance org anizations (HMOs) of The HMO Group took part in the project. A split-h alf design was used to test the standard and alternate survey forms co ncurrently. Surveys were administered by using the NCQA-recommended ma il methodology of survey and cover letter, reminder card, and second s urvey. Assuming a 50% response rate, a target of 400 responses (200 fo r each survey form) per HMO was planned. The window for responding was allowed to remain open 4 weeks beyond the mailing of the second surve y to achieve the desired response rate. A total of 4,056 responses wer e collected (2,022 for the standard form and 2,034 for the alternate f orm). It was found that the addition of ''No Experience'' and ''No Opi nion'' response options to the majority of satisfaction items reduced the random error associated with uninformed responses and produced sta tistically significant higher correlations with the global satisfactio n items relative to the standard form items. Only four of the eight Sh ort Form 12 summary scales, General Health, Reported Health Transition , Mental Health, and Social Functioning, were useful for adjusting dat a (covariation). The entire set of comorbidities (chronic disease chec klist) could be eliminated without losing significant data adjustment capability. The multiple linear regression models generated by using t he global satisfaction items on the alternate form had higher adjusted R-2 values than the standard-form models. The alternate-form item Ove rall Value correlated highly with cost items and general satisfaction items, making it a useful global satisfaction variable for predictive modeling.