'Televaluation' of clinical information systems: an integrative approach to assessing Web-based systems

The World Wide Web provides an unprecedented opportunity for widespread acc ess to health-care applications by both patients and providers. The develop ment of new methods for assessing the effectiveness and usability of these systems is becoming a critical issue. This paper describes the distance eva luation (i.e. 'televaluation') of emerging Web-based information technologi es. In health informatics evaluation, there is a need for application of ne w ideas and methods from the fields of cognitive science and usability engi neering. A framework is presented for conducting evaluations of health-care information technologies that integrates a number of methods, ranging from deployment of on-line questionnaires (and Web-based forms) to remote video -based usability testing of user interactions with clinical information sys tems. Examples illustrating application of these techniques are presented f or the assessment of a patient clinical information system (PatCIS), as wel l as an evaluation of use of Web-based clinical guidelines. Issues in desig ning, prototyping and iteratively refining evaluation components an discuss ed, along with description of a 'virtual' usability laboratory. (C) 2001 El sevier Science Ireland Ltd. All rights reserved.