Aj. Meade et Aa. Fernandez, SOLUTION OF NONLINEAR ORDINARY DIFFERENTIAL-EQUATIONS BY FEEDFORWARD NEURAL NETWORKS, Mathematical and computer modelling, 20(9), 1994, pp. 19-44
It is demonstrated, through theory and numerical examples, how it is p
ossible to directly construct a feedforward neural network to approxim
ate nonlinear ordinary differential equations without the need for tra
ining. The method, utilizing a piecewise linear map as the activation
function, is linear in storage, and the L(2) norm of the network appro
ximation error decreases monotonically with the increasing number of h
idden layer neurons. The construction requires imposing certain constr
aints on the values of the input, bias, and output weights, and the at
tribution of certain roles to each of these parameters. All results pr
esented used the piecewise linear activation function. However, the pr
esented approach should also be applicable to the use of hyperbolic ta
ngents, sigmoids, and radial basis functions.