RECTIFICATION OF CORRELATION BY A SIGMOID NONLINEARITY

We investigated the normalized autocovariance (correlation coefficient ) function of the output of an erf() function nonlinearity subject to non-zero mean Gaussian noise input. When the sigmoid is wide compared to the input, or the input mean is close to the midpoint of the sigmoi d, the output correlation coefficient function is very close to the in put correlation coefficient function. When the noise mean and variance are such that there is a significant probability of operating in the saturation region and the sigmoid is not too flat, the correlation coe fficient of the output function is less than that of the input. This d ifference is much greater when the correlation coefficient is negative than when it is positive. The sigmoid partially rectifies the correla tion coefficient function. The analysis does not depend on the spectra l properties of the input noise. All that is required is that the inpu t at times t and (t + tau) be jointly gaussian with the same mean and autocovariance. The analysis therefore applies equally well to the cas e of two identical sigmoids with jointly gaussian inputs. This correla tional rectification could help explain the parameter sensitivity of ' 'neural network'' models. If biological neurons share this property it could explain why few negative correlations between spike trains-have been observed.