Redundancy reduction revisited

Soon after Shannon defined the concept of redundancy it was suggested that it gave insight into mechanisms of sensory processing, perception, intellig ence and inference, Can we now judge whether there is anything in this idea , and can we see where it should direct our thinking? This paper argues tha t the original hypothesis was wrong in over-emphasizing the role of compres sive coding and economy in neuron numbers, but right in drawing attention t o the importance of redundancy. Furthermore there is a clear direction in w hich it now points, namely to the overwhelming importance of probabilities and statistics in neuroscience. The brain has to decide upon actions in a c ompetitive, chance-driven world, and to do this well it must know about and exploit the non-random probabilities and interdependences of objects and e vents signalled by sensory messages. These are particularly relevant for Ba yesian calculations of the optimum course of action. Instead of thinking of neural representations as transformations of stimulus energies, we should regard them as approximate estimates of the probable truths of hypotheses a bout the current environment, for these are the quantities required by a pr obabilistic brain working on Bayesian principles.