Emergence of symmetric, modular, and reciprocal connections in recurrent networks with Hebbian learning

While learning and development are well characterized in feed-forward netwo rks, these features are more difficult to analyze in recurrent networks due to the increased complexity of dual dynamics - the rapid dynamics arising from activation states and the slow dynamics arising from learning or devel opmental plasticity. We present analytical and numerical results that consi der dual dynamics in a recurrent network undergoing Hebbian learning with e ither constant weight decay or weight normalization. Starting from initiall y random connections, the recurrent network develops symmetric or near-symm etric connections through Hebbian learning. Reciprocity and modularity aris e naturally through correlations in the activation states. Additionally, we ight normalization may be better than constant weight decay for the develop ment of multiple attractor states that allow a diverse representation of th e inputs. These results suggest a natural mechanism by which synaptic plast icity in recurrent networks such as cortical and brainstem premotor circuit s could enhance neural computation and the generation of motor programs.