REDUNDANCY REDUCTION AND INDEPENDENT COMPONENT ANALYSIS - CONDITIONS ON CUMULANTS AND ADAPTIVE APPROACHES

In the context of both sensory coding and signal processing, building factorized codes has been shown to be an efficient strategy. In a wide variety of situations, the signal to be processed is a linear mixture of statistically independent sources. Building a factorized code is t hen equivalent to performing blind source separation. Thanks to the li near structure of the data, this can be done, in the language of signa l processing, by finding an appropriate linear filter, or equivalently , in the language of neural modeling, by using a simple feedforward ne ural network. In this article, we discuss several aspects of the sourc e separation problem. We give simple conditions on the network output that, if satisfied, guarantee that source separation has been obtained . Then we study adaptive approaches, in particular those based on redu ndancy reduction and maximization of mutual information. We show how t he resulting updating rules are related to the BCM theory of synaptic plasticity. Eventually we briefly discuss extensions to the case of no nlinear mixtures. Throughout this article, we take care to put into pe rspective our work with other studies on source separation and redunda ncy reduction. In particular we review algebraic solutions, pointing o ut their simplicity hut also their drawbacks.