MACROSCOPIC PROPERTIES OF THE COST FUNCTION OF A FEEDFORWARD NEURAL-NETWORK PRIOR TO TRAINING

By making simple assumptions regarding the nodal potentials we have be en able to obtain analytic expressions for the mean and standard devia tion of the cost-function values of a feed-forward multilayer network, with continuous activation units, prior to training. We have also obt ained means of the derivatives, with respect to the weights and biases , of the cost function. The expressions have been used to obtain syste matic estimates of the learning rate required for back-propagation tra ining. The results are exemplified using an 8-3-8 encoder network.