The use of cognitive testing to develop and evaluate CAHPS (TM) 1.0 core survey items

OEJECTIVES. The main goal of the Consumer Assessments of Health Plans Study (CAHPS(TM)) is to develop an integrated set of tested, standardized survey s to obtain meaningful information from health plan enrollees about their e xperiences. The CAHPS(TM) project benefits from the complementary strengths of psychometric and cognitive testing. METHODS. The CAHPS(TM) team conducted 150 cognitive interviews across three organizations in different geographic locations using multiple interview m ethods with different consumer populations. This article explains how cogni tive testing was used in the CAHPS(TM) survey development process and share s the main findings from the cognitive interviews. RESULTS. A modified report format is more appropriate when asking about spe cific aspects of plan enrollees' experiences, whereas a rating format is us eful for asking about overall assessments. Specifying a longer reference pe riod is different explains multiple interview methods consumer populations. This preferable to asking about the most recent visit when capturing exper iences with care, because some respondents get frustrated when they cannot include experiences other than the most recent visit. Explicit screeners an d tailored inapplicable response categories are beneficial in mail question naires, so people know that they should not answer questions about which th ey have no relevant experience. CONCLUSION. Cognitive testing was integral in the development and refinemen t of the CAHPS(TM) instrument. The cognitive testing findings contributed t o an improved instrument that should capture consumers' health care and pla n experiences with less response error than one not subjected to such testi ng. The cognitive testing process and findings can be useful to other resea rchers with similar survey development goals.