The limits of counting accuracy in distributed neural representations

Learning about a causal or statistical association depends on comparing fre quencies of joint occurrence with frequencies expected from separate occurr ences, and to do this, events must somehow be counted. Physiological mechan isms can easily generate the necessary measures if there is a direct, one-t o-one relationship between significant events and neural activity, but if t he events are represented across cell populations in a distributed manner, the counting of one event will be interfered with by the occurrence of othe rs. Although the mean interference can be allowed for, there is inevitably an increase in the variance of frequency estimates that results in the need for extra data to achieve reliable learning. This lowering of statistical efficiency (Fisher, 1925) is calculated as the ratio of the minimum to actu al variance of the estimates. We define two neural models, based on presyna ptic and Hebbian synaptic modification, and explore the effects of sparse c oding and the relative frequencies of events on the efficiency of frequency estimates. High counting efficiency must be a desirable feature of biologi cal representations, but the results show that the number of events that ca n be counted simultaneously with 50% efficiency is fewer than the number of cells or 0.1-0.25 of the number of synapses (on the two models)-many fewer than can be unambiguously represented. Direct representations would lead t o greater counting efficiency, but distributed representations have the ver satility of detecting and counting many unforeseen or rare events. Efficien t counting of rare but important events requires that they engage more acti ve cells than common or unimportant ones. The results suggest reasons that representations in the cerebral cortex appear to use extravagant numbers of cells and modular organization, and they emphasize the importance of neuro nal trigger features and the phenomena of habituation and attention.