NATURAL IMAGE STATISTICS AND EFFICIENT CODING

Natural images contain characteristic statistical regularities that se t them apart from purely random images. Understanding what these regul arities are can enable natural images to be coded more efficiently. In this paper, we describe some of the forms of structure that are conta ined in natural images, and we show how these are related to the respo nse properties of neurons at early stages of the visual system. Many o f the important forms of structure require higher-order (i.e. more tha n linear, pairwise) statistics to characterize, which makes models bas ed on linear Hebbian learning, or principal components analysis, inapp ropriate for finding efficient codes for natural images. We suggest th at a good objective for an efficient coding of natural scenes is to ma ximize the sparseness of the representation, and we show that a networ k that learns sparse codes of natural scenes succeeds in developing lo calized, oriented, bandpass receptive fields similar to those in the m ammalian striate cortex.