INFORMATION-PROCESSING BY A NOISY BINARY CHANNEL

We study the information processing properties of a binary channel rec eiving data from a Gaussian source. A systematic comparison with linea r processing is made. A remarkable property of the binary sytem is tha t, as the ratio alpha between the number of output and input units inc reases, binary processing becomes equivalent to linear processing with a quantization output noise that depends on alpha. In this regime, wh ich holds up to O(alpha(-4)), information processing occurs as if popu lations of alpha binary units cooperate to represent one alpha-bit out put unit. Unsupervised learning of a noisy environment by optimization of the parameters of the binary channel is also considered.