We study associative and storage memories for memory traces of size N
and aim to establish that both the size of the system (as measured by,
e.g., the number of nodes in a network) can be of order N and the num
ber of traces consistent with reliable operation can be exponentially
large in N, so that a positive capacity (in bits per node) can be achi
eved. It is well known that, if the traces are generated as M random v
ectors, then reliability imposes a linear bound on M, in that it impli
es an upper bound on the asymptotic (large N) value of alpha = M/N. Fo
r the noise-free Hopfield net this critical bound is about 0.138. We s
how that, if superposition of traces is allowed, so that the M given t
races constitute the random basis of a linear code, then exponential m
emory size and a positive capacity can be achieved. However, there is
still a critical upper bound on the basis rate alpha = M/N, implied no
w, not by the condition of reliability: but by the necessity that the
recursion realising the calculation should be stable. For our model we
determine this critical value exactly as alpha(c)=3-root 8=0.172. Our
model is based upon inference concepts and differs in slight but impo
rtant respects from the Hopfield model. We do not use replica methods,
but appeal to a generalised version of the Wigner semi-circle theorem
on the asymptotic distribution of eigenvalues. (C) 1997 Elsevier Scie
nce Ltd. All rights reserved.