THE ROLE OF RULES AND EXAMPLES IN THE PROCESS OF KNOWLEDGE ACQUISITION IN DIRECT CLASSIFICATION TASKS

Shells provide a means for experts to easily develop expert systems fo r their area of expertise. However, rule bases need to be complete and free of contradictions. A set of 30 subjects, unfamiliar with shells except for initial orientation and training, were asked to develop a s ystem for their personal preferences for a decision problem. The resul ts of these systems were analyzed, leading to a number of conclusions. First, three types of rules used by the subjects were identified. Cut off rules reflect preemptive treatment of decision rules. Examples ref lect an attempt to enumerate all combinations of decision factors. Com pensatory rules reflect attempts to balance trade-offs among the relat ive performance of decision cases. The implications of using these thr ee types of rules are evaluated. Subjects validated their systems on a test bank of 18 cases. Subject responses to the impact of these test cases were evaluated, revealing that they thought that the test cases yielded more complete systems. Posttest evaluation of their systems fo r completeness and consistency also revealed that the sytstems still i ncluded significant gaps in rules. We conclude that computer aids to a ssist experts need to include means to assure consistency and complete ness of knowledge bases. Further, at least some compensatory rules sho uld be included for those cases that involve trade-offs.