EVALUATION OF A MEDICAL DIAGNOSIS SYSTEM USING SIMULATOR TEST SCENARIOS

This paper describes an informal but systematic method for how to test and verify a knowledge-based system in a large and open-ended medical target domain. The system used is Guardian, an intelligent system for monitoring and diagnosis of post-cardiac surgery patients in an inten sive-care unit. The knowledge base is tested and verified by running t he system on a series of realistic test scenarios, both with an embedd ed simulator and with an external simulation system. The same scenario s are presented to human test subjects, making it possible to compare and analyze the performance of the knowledge-based system with that of human physicians. The use of simulators instead of clinical data also means that it is possible to test crucial scenarios which occur seldo m in medical practice. Our results show that a system like Guardian mi ght indeed be useful in medical care. (C) 1997 Elsevier Science B.V.