Assessment of general practitioners by video observation of communicative and medical performance in daily practice: issues of validity, reliability and feasibility

Objectives To develop a video assessment method for General Practitioners ( GPs) by analysing issues of validity, reliability and feasibility of observ ation of videotaped regular consultations. Design In a cross-sectional study consultations of 93 GPs were video record ed in the practice during 1 week. The GPs registered consultation and patie nt data in a logbook; 16 consultations per GP were selected using preset cr iteria. The quality of communicative and medical performance of these consu ltations was assessed by GP observers with a validated instrument. The vali dity of the procedure was evaluated by checking the content of each GP's sa mple using specific sample criteria. Selection bias was estimated by multip le regression analysis, with sample characteristics as independent variable s and scores on communication and medical performance as dependent variable s. The influence of observation on GPs and patients was assessed by a quest ionnaire. Generalizability theory was used to estimate reliability. Feasibi lity was assessed by conducting a questionnaire, by keeping accounts, and b y checking the technical quality of the videotaped consultations. Setting Universities of Nijmegen and Maastricht, The Netherlands. Subjects General Practitioners (GPs). Results The domain of general practice was well covered in the samples; con tent validity was satisfactory. With regard to the sample characteristics, only the total duration of consultations appeared to correlate significantl y with both the score on communication and the score on medical performance . A majority (71%) of GPs reported not being influenced by the observation, except in the first cases, and recognizing their usual daily performance i n the videotaped consultations. An acceptable level of reliability was reac hed after 2.5 hours of observation, i.e. 12 cases by a single observer. The method was well accepted by both GPs and patients. The costs were pound 25 0 per GP. Conclusions Video assessment of GPs in daily practice according to the proc edures described is a valid and reliable method, one which is useful for ed ucation and quality improvement. There is a trade-off between feasibility o n one hand and validity, reliability and credibility on the other hand. Com pared to investments in observation methods in standardized settings, the c osts of video observation of GPs' actual performance are acceptable.