DOES IT MATTER WHOM AND HOW YOU ASK - INTER-RATER AND INTRA-RATER AGREEMENT IN THE ONTARIO HEALTH SURVEY

A large amount of information in the 1990 Ontario Health Survey (OHS) was collected from proxy respondents using questions administered in f ace-to-face interviews. Can this type of information represent candid self-reported measures of health status? Inter-rater agreement was ass essed using Cohen's kappa statistic for responses to questions that we re answered both by individuals about themselves and by proxies on the ir behalf. Intra-rater agreement, assessing the effect of mode of surv ey administration (in-person interviews versus self-completed written questionnaires) on the responses, was also investigated using the kapp a statistic. We conclude that: (1) proxy responses in the OHS for impa irments of emotion and pain are not reliable indicates of self-respons e (kappa < 0.32) because proxy respondents consistently under-report t he burden of morbidity; (2) levels of morbidity reported by subjects t o interviewer-administered questionnaires may underestimate morbidity, relative to morbidity reported by subjects using self-administered qu estionnaires completed in privacy. We also hypothesize that the relati ve magnitudes of inaccuracy introduced by interviewer administration r elative to proxy reporting depends on the phenomenon being measured. W hen assessing pain, mode of administration is quantitatively a more im portant source of disagreement than type of respondent. Copyright (C) 1997 Elsevier Science Inc.