Generalized radial basis function networks for classification and novelty detection: self-organization of optimal Bayesian decision

By adding reverse connections from the output layer to the central layer it is shown how a generalized radial basis functions (GRBF) network can self- organize to form a Bayesian classifier, which is also capable of novelty de tection. For this purpose, three stochastic sequential learning rules are i ntroduced from biological considerations which pertain to the centers, the shapes, and the widths of the receptive fields of the neurons and allow a j oint optimization of all network parameters. The rules are shown to generat e maximum-likelihood estimates of the class-conditional probability density functions of labeled data in terms of multivariate normal mixtures. Upon c ombination with a hierarchy of deterministic annealing procedures, which im plement a multiple-scale approach, the learning process can avoid the conve rgence problems hampering conventional expectation-maximization algorithms. Using an example from the field of speech recognition, the stages of the l earning process and the capabilities of the self-organizing GRBF classifier are illustrated. (C) 2000 Elsevier Science Ltd. All rights reserved.