Emergence of phase- and shift-invariant features by decomposition of natural images into independent feature subspaces

Olshausen and Field (1996) applied the principle of independence maximizati on by sparse coding to extract features from natural images. This leads to the emergence of oriented linear filters that have simultaneous localizatio n in space and in frequency, thus resembling Gabor functions and simple cel l receptive fields. In this article, we show that the same principle of ind ependence maximization can explain the emergence of phase- and shift-invari ant features similar to those found in complex cells. This new kind of emer gence is obtained by maximizing the independence between norms of projectio ns on linear subspaces (instead of the independence of simple linear filter outputs). The norms of the projections on such "independent feature subspa ces" then indicate the values of invariant features.