Jp. Nadal et N. Parga, REDUNDANCY REDUCTION AND INDEPENDENT COMPONENT ANALYSIS - CONDITIONS ON CUMULANTS AND ADAPTIVE APPROACHES, Neural computation, 9(7), 1997, pp. 1421-1456
In the context of both sensory coding and signal processing, building
factorized codes has been shown to be an efficient strategy. In a wide
variety of situations, the signal to be processed is a linear mixture
of statistically independent sources. Building a factorized code is t
hen equivalent to performing blind source separation. Thanks to the li
near structure of the data, this can be done, in the language of signa
l processing, by finding an appropriate linear filter, or equivalently
, in the language of neural modeling, by using a simple feedforward ne
ural network. In this article, we discuss several aspects of the sourc
e separation problem. We give simple conditions on the network output
that, if satisfied, guarantee that source separation has been obtained
. Then we study adaptive approaches, in particular those based on redu
ndancy reduction and maximization of mutual information. We show how t
he resulting updating rules are related to the BCM theory of synaptic
plasticity. Eventually we briefly discuss extensions to the case of no
nlinear mixtures. Throughout this article, we take care to put into pe
rspective our work with other studies on source separation and redunda
ncy reduction. In particular we review algebraic solutions, pointing o
ut their simplicity hut also their drawbacks.