COGNITIVE EVALUATION OF DECISION-MAKING PROCESSES AND ASSESSMENT OF INFORMATION TECHNOLOGY IN MEDICINE

This paper describes cognitive methods for analyzing medical decision making and evaluating medical information systems. The overall approac h focuses on understanding the processes involved in the decision maki ng and reasoning of health care workers, both with and without the use of information technologies. The issue of developing appropriate eval uation tools, for use in the design and analysis of medical informatio n systems is considered to be of great importance. However, convention al methods are limited in their ability to identify and characterize t he effects of information technology on the cognitive processes involv ed in decision making and reasoning. In this paper a range of methods are described involving video recording for collecting data on the use of information systems. The techniques described allow for the collec tion of an integrated data set consisting of transcripts of health car e workers as they 'think aloud' in interacting with a medical system, along with complete video records of user-computer interaction. In add ition, the methods can be extended to allow for the collection of proc ess data from video recording of systems in actual clinical and emerge ncy situations. The use bf a variety of approaches, borrowing from res earch in cognitive science, is discussed. The development and applicat ion of these evaluation methods within the Canadian Centres of Excelle nce network HEALNet is subsequently described. Finally, implications f or the development and evaluation of medical information systems are c onsidered. (C) 1998 Elsevier Science Ireland Ltd. All rights reserved.