VERIFICATION OF UNCERTAIN KNOWLEDGE-BASED SYSTEMS - AN EMPIRICAL VERIFICATION APPROACH

A number of different tests and approaches are developed to determine the existence of potential anomalies in rule-based systems that employ MYCIN uncertainty factors (weights). First, the distribution of weigh ts is compared to other systems' distributions and weights are investi gated as to their individual meanings, to determine whether any weight s are unusual. Second, there is increasing evidence that people are no t ''good'' at developing weights on rules, building in symmetries and redundancies that signal ''usual'' assumptions about the underlying pr obabilities. Accordingly, weight symmetries generated from rule pairs are analyzed to determine the existence of anomalies. Third, typically rule-based tools have been developed for application in specific doma ins, such as medicine. Unique aspects of those domains may limit appli cation of the tools to other domains. Finally, ad hoc, rule-based appr oaches are suboptimal, and alternative formal probability approaches, such as Bayes' nets, more fully specify the probabilistic nature of kn owledge. The paper is part of the empirical verification literature, w here verification is done on an actual system and the system provides data that indicates the kinds of anomalies that can be expected. A cas e study is used to illustrate each of the verification tests and conce rns.