DOMAINS OF SOLUTIONS AND REPLICA SYMMETRY-BREAKING IN MULTILAYER NEURAL NETWORKS

The relationship between the geometrical structure of weight space and replica symmetry breaking (RSB) in multilayer neural networks is stud ied using a toy model. The distribution of sizes of the disconnected d omains of solution space is computed analytically and compared to the RSB calculation of the Gardner volume. We are able to show explicitly that ergodicity breaking and RSB are not equivalent. Repeating these c alculations using the cavity approach allows us to interpret the geome trical meaning of a RSB ansatz.