APPROXIMATION BY RIDGE FUNCTIONS AND NEURAL NETWORKS

We investigate the efficiency of approximation by linear combinations of ridge functions in the metric of L-2(B-d) with B-d the unit ball in R-d. If X-n is an n-dimensional linear space of univariate functions in L-2(I), I = [-1; 1], and Omega is a subset of the unit sphere S-d-1 in R-d of cardinality m, then the space Y-n := span {r(x . xi) : r is an element of X-n, omega is an element of Omega} is a linear space of ridge functions of dimension less than or equal to mn. We show that i f X-n provides order of approximation O(n(-r)) for univariate function s with r derivatives in L-2(I), and Omega are properly chosen sets of cardinality O(n(d-1)), then Y-n will provide approximation of order O( n(-r-d/2+1/2)) for every function f is an element of L-2(B-d) with smo othness of order r + d/2 - 1/2 in L-2(B-d). Thus, the theorems we obta in show that this form of ridge approximation has the same efficiency of approximation as other more traditional methods of multivariate app roximation such as polynomials, splines, or wavelets. The theorems we obtain can be applied to show that a feed-forward neural network with one hidden layer of computational nodes given by certain sigmoidal fun ction sigma will also have this approximation efficiency. Minimal requ irements are made of the sigmoidal functions and in particular our res ults hold for the unit-impulse function sigma = chi([0, infinity)).