MULTILAYER FEEDFORWARD NETWORKS WITH A NONPOLYNOMIAL ACTIVATION FUNCTION CAN APPROXIMATE ANY FUNCTION

Several researchers characterized the activation function under which multilayer feedforward networks can act as universal approximators. We show that most of all the characterizations that were reported thus f ar in the literature are special cases of the following general result : A standard multilayer feedforward network with a locally bounded pie cewise continuous activation function can approximate any continuous f unction to any degree of accuracy if and only if the network's activat ion function is not a polynomial. We also emphasize the important role of the threshold, asserting that without it the last theorem does not hold.