DISCERN: an instrument for judging the quality of written consumer health information on treatment choices

Objective-To develop a short instrument, called DISCERN, which will enable patients and information providers to judge the quality of written informat ion about treatment choices. DISCERN will also facilitate the production of new, high quality, evidence-based consumer health information. Design-An expert panel, representing a range of expertise in consumer healt h information, generated criteria from a random sample of information for t hree medical conditions with varying degrees of evidence: myocardial infarc tion, endometriosis, and chronic fatigue syndrome. A draft instrument, base d on this analysis, was tested by the panel on a random sample of new mater ial for the same three conditions. The panel re-drafted the instrument to t ake account of the results of the test. The DISCERN instrument was finally tested by a national sample of 15 information providers and 13 self help gr oup members on a random sample of leaflets from 19 major national self help organisations. Participants also completed an 8 item questionnaire concerning the face and content validity of the instrument. Results-Chance corrected agreement (we ighted kappa) for the overall quality rating was kappa=0.53 (95% CI kappa=0 .48 to kappa=0.59) among the expert panel, kappa=0.40 (95% CI kappa=0.36 to kappa=0.43) among information providers, and kappa=0.23 (95% CI kappa=0.19 to kappa=0.27) among self help group members. Higher agreement levels were associated with experience of using the instrument and with professional k nowledge of consumer health information. Levels of agreement varied across individual items on the instrument, reflecting the need for subjectivity in rating certain criteria. The trends in levels of agreement were similar am ong all groups. The final instrument consisted of 15 questions plus an over all quality rating. Responses to the questionnaire after the final testing revealed the instrument to have good face and content validity and to be ge nerally applicable. Conclusions-DISCERN is a reliable and valid instrument for judging the qual ity of written consumer health information. While some subjectivity is requ ired for rating certain criteria, the findings demonstrate that the instrum ent can be applied by experienced users and providers of health information to discriminate between publications of high and low quality. The instrume nt will also be of benefit to patients, though its use will be improved by training.