We investigate the consequences of maximizing information transfer in
a simple neural network (one input layer, one output layer), focusing
on the case of nonlinear transfer functions. We assume that both recep
tive fields (synaptic efficacies) and transfer functions can be adapte
d to the environment. The main result is that, for bounded and inverti
ble transfer functions, in the case of a vanishing additive output noi
se, and no input noise, maximization of information (Linsker's infomax
principle) leads to a factorial code - hence to the same solution as
required by the redundancy-reduction principle of Barlow. We also show
that this result is valid for linear and, more generally, unbounded,
transfer functions, provided optimization is performed under an additi
ve constraint, i.e. which can be written as a sum of terms, each one b
eing specific to one output neuron. Finally, we study the effect of a
non-zero input noise. We find that, to first order in the input noise,
assumed to be small in comparison with the (small) output noise, the
above results are still valid, provided the output noise is uncorrelat
ed from one neuron to the other.