Evaluation of clinical information systems. What can be evaluated and whatcannot?

The evaluation of clinical information systems is essential as they are inc reasingly used in clinical routine and may even influence patient outcome o n the basis of reminder functions and decision support. Therefore we try to answer three questions in this paper: what to evaluate; how to evaluate: h ow to interpret the results. Those key questions lead to the discussion of goals, methods and results of evaluation studies in a common context. We wi ll compare the objectivist and the subjectivist evaluation approach and ill ustrate the evaluation process itself in some detail, discussing different phases of software development and potential evaluation techniques in each phase. We use four different practical examples of evaluation studies that were conducted in various settings to demonstrate how defined evaluation go als may be achieved with a limited amount of resources. This also illustrat es advantages, limitations and costs of the different evaluation methods an d techniques that may be used when evaluating clinical information systems.