SIMULATING THE INTEGRATED CRITIQUING SYSTEM

Objective: To investigate factors that determine the feasibility and e ffectiveness of a critiquing system for asthma/COPD that will be integ rated with a general practitioner's (GP's) information system. Design: A simulation study. Four reviewers, playing the role of the computer, generated critiquing comments and requests for additional information an six electronic medical records of patients with asthma/COPD. Three GPs who treated the patients, playing users, assessed the comments an d provided missing information when requested. The GPs were asked why requested missing information was unavailable. The reviewers reevaluat ed their comments after receiving requested missing information, Measu rements: Descriptions of the number and nature of critiquing comments and requests for missing information. Assessment by the GPs of the cri tiquing comments in terms of agreement with each comment and judgment of its relevance, both on a five-point scale. Analysis of causes for t he (un-)availability of requested missing information. Assessment of t he impact of missing information on the generation of critiquing comme nts. Results: Four reviewers provided 74 critiquing comments on 87 vis its in six medical records. Most were about prescriptions (n = 28) and the GPs' workplans (n = 27). The GPs valued comments about diagnostic s the most. The correlation between the GPs' agreement and relevance s cores was 0.65. However, the GPs' agreements with prescription comment s (complete disagreement, 31.3%; disagreement, 20.0%; neutral, 13.8%; agreement, 17.5%; complete agreement, 17.5%) differed from their judgm ents of these comments' relevance (completely irrelevant, 9.0%; irrele vant, 24.4%; neutral, 24.4%; relevant, 32.1%; completely relevant, 10. 3%). The GPs were able to provide answers to 64% of the 90 requests fo r missing information. Reasons available information had not been reco rded were: the GPs had not recorded the information explicitly; they h ad assumed it to be common knowledge; it was available elsewhere in th e record. Reasons information was unavailable were: the decision had b een made by another; the GP had not recorded the information. The revi ewers left 74% of the comments unchanged after receiving requested mis sing information. Conclusion: Human reviewers can generate comments ba sed on information currently available in electronic medical records o f patients with asthma/COPD. The GPs valued comments regarding rite di agnostic process the most. Although they judged prescription comments relevant, they often strongly disagreed with them, a discrepancy that poses a challenge for the presentation of critiquing comments for the future critiquing system. Requested additional information that was pr ovided by the GPs led to few changes. Therefore, as system developers faced with the decision to build an integrated, non-inquisitive or an inquisitive critiquing system, the authors choose the former.