HOW VALID ARE MEDICAL RECORDS AND PATIENT QUESTIONNAIRES FOR PHYSICIAN PROFILING AND HEALTH-SERVICES RESEARCH - A COMPARISON WITH DIRECT OBSERVATION OF PATIENT VISITS

OBJECTIVES. This study was designed to determine the optimal nonobserv ational method of measuring the delivery of outpatient medical service s. METHODS. AS part of a multimethod study of the content of primary c are practice, research nurses directly observed consecutive patient vi sits to 138 practicing family physicians. Data on services delivered w ere collected using a direct observation checklist, medical record rev iew, and patient exit questionnaires. For each medical service, the se nsitivity, specificity, and Kappa statistic were calculated for medica l record review and patient exit questionnaires compared with direct o bservation. Interrater reliability among eight research nurses was cal culated using the Kappa statistic for a separate sample of videotaped visits and medical records. RESULTS. Visits by 4,454 patients were obs erved. Exit questionnaires were returned by 75% of patients. Research nurse interrater reliabilities were generally high. The specificity of oth the medical record and the patient exit questionnaire was high fo r most services. The sensitivity of the medical record was low for mea suring health habit counseling and moderate for physical examination, laboratory testing, and immunization. The patient exit questionnaire s howed moderate to high sensitivity for health habit counseling and imm unization and variable sensitivity for physical examination and labora tory services. CONCLUSIONS. The validity of the medical record and pat ient questionnaire for measuring delivery of different health services varied with the service. This report can be used to choose the optima l nonobservational method of measuring the delivery of specific ambula tory medical services for research and physician profiling and to inte rpret existing health services research studies using these common mea sures.