Probability density estimation using artificial neural networks

We present an approach for the estimation of probability density functions (pdf) given a set of observations. It is based on the use of feedforward mu ltilayer neural networks with sigmoid hidden units. The particular characte ristic of the method is that the output of the network is not a pdf, theref ore, the computation of the network's integral is required. When this integ ral cannot be performed analytically, one is forced to resort to numerical integration techniques. It turns out that this is quite tricky when coupled with subsequent training procedures. Several modifications of the original approach (Modha and Fainman, 1994) are proposed, most of them related to t he numerical treatment of the integral and the employment of a preprocessin g phase where the network parameters are initialized using supervised train ing. Experimental results using several test problems indicate that the pro posed method is very effective and in most cases superior to the method of Gaussian mixtures. (C) 2001 Elsevier Science B.V. All rights reserved.