STATE-OF-THE-ART IN AUTOMATED VALIDATION OF KNOWLEDGE-BASED SYSTEMS

Validation of Knowledge-Based Systems (KBS) is an important aspect of the overall KBS development process, which aims to assure the system's ability to reach correct conclusions. The objective of this paper is to discuss the desirable functionality of an automated validation tool and to provide a survey of existing methods and tools supporting that functionality. The scope of our discussion is limited to validating t he level of performance of the KBS as a problem solver, since this is the aspect in which KBS differ most from conventional software; more c onventional aspects of system evaluation, such as assessing the ''usab ility'' of the system, are not covered. Automated validation tools are considered in two categories: dynamic and static. Dynamic validation tools are those that measure and, in some cases, refine the level of p erformance of a KBS using a suite of test cases. Use of such tools ass umes that an adequate set of real test cases is available. Static vali dation tools are used to create test cases by making use of domain kno wledge already embodied in the KBS or meta-knowledge. Such tools are u sed when an inadequate set of test cases is available.