EVALUATION OF VERIFICATION TOOLS FOR KNOWLEDGE-BASED SYSTEMS

Validation has emerged as a significant problem in the development of knowledge-based systems (KBS). Verification of KBS correctness and com pleteness has been cited as one of the most difficult aspects of valid ation. A number of software tools have been developed to perform such verification, but none of these are in widespread use. One of the reas ons for this is that little quantitative evidence exists to demonstrat e the effectiveness of the tools. This paper presents an experimental study of three KBS verification tools: a consistency checker, a comple teness checker and a testing tool(for correctness). The tools are eval uated on their ability to reveal plausible faults seeded into a comple x, realistic KBS application. The cost of using the tools is also meas ured. It is shown that each tool is independently effective at detecti ng certain kinds of fault and that the capabilities of the tools are c omplementary-a result not revealed by previous studies. (C) 1997 Acade mic Press Limited.