Template-based procedures for neural network interpretation

Although neural networks often achieve impressive learning and generalizati on performance, their internal workings are typically all but impossible to decipher. This characteristic of the networks, their opacity, is one of th e disadvantages of connectionism compared to more traditional, rule-oriente d approaches to artificial intelligence. Without a thorough understanding o f the network behavior, confidence in a system's results is lowered, and th e transfer of learned knowledge to other processing systems - including hum ans - is precluded. Methods that address the opacity problem by casting net work weights in symbolic terms are commonly referred to as rule extraction techniques. This work describes a principled approach to symbolic rule extr action from standard multilayer feedforward networks based on the notion of weight templates, parameterized regions of weight space corresponding to s pecific symbolic expressions. With an appropriate choice of representation, we show how template parameters may be efficiently identified and instanti ated to yield the optimal match to the actual weights of a unit. Depending on the requirements of the application domain. the approach can accommodate n-ary disjunctions and conjunctions with O(k) complexity, simple n-of-m ex pressions with O(k(2)) complexity, or more general classes of recursive n-o f-m expressions with O(k(L+2)) complexity, where ii is the number of inputs to an unit and L the recursion level of the expression class. Compared to other approaches in the literature, our method of rule extraction offers be nefits in simplicity, computational performance, and overall flexibility. S imulation results on a variety of problems demonstrate the application of o ur procedures as well as the strengths and the weaknesses of our general ap proach. (C) 1999 Published by Elsevier Science Ltd. All rights reserved.