Sensitivity, specificity, reliability, and clinical validity of provider-reported symptoms: A comparison with self-reported symptoms

Background: If symptoms are to be recognized and effectively addressed in c linical research, they must be collected using sensitive, specific, reliabl e, and clinically meaningful methods. Objective: To perform a comparison of self-administered symptom survey data with data from conventional provider-reports. Design/methods: Secondary data analysis of AIDS Clinical Trials Group Study 081 (ACTG 081), a randomized trial taking place in 33 sites comparing thre e approaches to prophylaxis for Pneumocystis carinii-related pneumonia that found no difference among treatment arms. The study was performed on 842 s ubjects with advanced HIV infection. No intervention was undertaken as a re sult of this study. ACTG 081 included data on functional status, global qua lity of life and survival, and two methods of symptom measurement: an open- ended, provider-reported symptom assessment (provider-report) and a self-ad ministered symptom survey (self-report). Agreement was measured using kappa scores. Sensitivity and specificity were calculated using self-report as t he standard. Reliability was measured by intersite variation and test-retes t reliability (8 weeks later). Clinical validity was evaluated by testing e xpected associations with functional status, global quality of life, and su rvival. Results: Symptom data were available for 808 patients (96%). Patient and pr ovider agreement was poor (mean kappa, 0.14; range, 0.07-0.25). Compared wi th self-report, providers underreported the presence and severity of sympto ms (mean symptom count, 5.2 versus 1.3; mean severity score, 1.3 versus 0.7 4). provider-report demonstrated greater variability by site (R-2 associate d with site, 0.02 versus 0.16) and poorer test-retest reliability (mean kap pa, 0.34 versus 0.25). Provider-report severity scores were less strongly a ssociated than were self-report with functional status (chi(2), 252 versus 80) global quality of life (R-2 for model, 0.57 versus 0.15), and survival (chi(2), 38 versus 24). Self-reported symptom severity was strongly correla ted to patient-reported global quality of life (rho, 0.75; p < .0001). Conclusions: Provider-reported symptoms as currently collected within the A CTG are less sensitive and reproducible than a self-administered symptom su rvey. Provider reported severity scores are also more weakly associated wit h functional status, global quality of life, and survival. A self-reported symptom survey may provide a better method of symptom measurement for HIV r esearch.