GENERATIVE MODELS FOR DISCOVERING SPARSE DISTRIBUTED REPRESENTATIONS

We describe a hierarchical, generative model that can be viewed as a n onlinear generalization of factor analysis and can be implemented in a neural network. The model uses bottom-up, top-down and lateral connec tions to perform Bayesian perceptual inference correctly Once perceptu al inference has been performed the connection strengths can be update d using a very simple learning rule that only requires locally availab le information. We demonstrate that the network learns to extract spar se, distributed, hierarchical representations.