Although neural networks often achieve impressive learning and generalizati
on performance, their internal workings are typically all but impossible to
decipher. This characteristic of the networks, their opacity, is one of th
e disadvantages of connectionism compared to more traditional, rule-oriente
d approaches to artificial intelligence. Without a thorough understanding o
f the network behavior, confidence in a system's results is lowered, and th
e transfer of learned knowledge to other processing systems - including hum
ans - is precluded. Methods that address the opacity problem by casting net
work weights in symbolic terms are commonly referred to as rule extraction
techniques. This work describes a principled approach to symbolic rule extr
action from standard multilayer feedforward networks based on the notion of
weight templates, parameterized regions of weight space corresponding to s
pecific symbolic expressions. With an appropriate choice of representation,
we show how template parameters may be efficiently identified and instanti
ated to yield the optimal match to the actual weights of a unit. Depending
on the requirements of the application domain. the approach can accommodate
n-ary disjunctions and conjunctions with O(k) complexity, simple n-of-m ex
pressions with O(k(2)) complexity, or more general classes of recursive n-o
f-m expressions with O(k(L+2)) complexity, where ii is the number of inputs
to an unit and L the recursion level of the expression class. Compared to
other approaches in the literature, our method of rule extraction offers be
nefits in simplicity, computational performance, and overall flexibility. S
imulation results on a variety of problems demonstrate the application of o
ur procedures as well as the strengths and the weaknesses of our general ap
proach. (C) 1999 Published by Elsevier Science Ltd. All rights reserved.