SPARSE CODING WITH AN OVERCOMPLETE BASIS-SET - A STRATEGY EMPLOYED BYV1

The spatial receptive fields of simple cells in mammalian striate cort ex have been reasonably well described physiologically and can be char acterized as being localized, oriented, and bandpass, comparable with the basis functions of wavelet transforms, Previously, we have shown t hat these receptive field properties may be accounted for in terms of a strategy for producing a sparse distribution of output activity in r esponse to natural images, Here, in addition to describing this work i n a more expansive fashion, we examine the neurobiological implication s of sparse coding, Of particular interest is the case when the code i s overcomplete-i.e., when the number of code elements is greater than the effective dimensionality of the input space, Because the basis fun ctions are non-orthogonal and not linearly independent of each other, sparsifying the code will recruit only those basis functions necessary for representing a given input, and so the input-output function will deviate from being purely linear, These deviations from linearity pro vide a potential explanation for the weak forms of non-linearity obser ved in the response properties of cortical simple cells, and they furt her make predictions about the expected interactions among units in re sponse to naturalistic stimuli. (C) 1997 Elsevier Science Ltd.