FINDING COMPACT AND SPARSE-DISTRIBUTED REPRESENTATIONS OF VISUAL IMAGES

Some recent work has investigated the dichotomy between compact coding using dimensionality reduction and sparse-distributed coding in the c oncert of understanding biological information processing. We introduc e an artificial neural network which self-organizes on the basis of si mple Hebbian learning and negative feedback of activation and show tha t it is capable both of forming compact codings of data distributions and of identifying filters most sensitive to sparse-distributed codes. The network is extremely simple and its biological relevance is inves tigated via its response to a set of images which are typical of every day life. However, an analysis of the network's identification of the filter for sparse coding reveals that this coding may not be globally optimal and that there exists an innate limiting factor which cannot b e transcended.