ARE ANALOG NEURAL NETWORKS BETTER THAN BINARY NEURAL NETWORKS

In this paper, we study the problem of maximizing an objective functio n over the discrete set {-1, 1}(n) using a neural network. It is now k nown that a binary (two-state) Hopfield network can take, in the worst case, an exponential number of time steps to find even a local maximu m of the objective function. In this paper, we carry this argument fur ther by studying the radius of attraction of the global maxima of the objective function. If a binary neural network is used, in general the re is no guarantee that a global maximum has a nonzero radius of attra ction. In other words, even if the optimization process is started off with the neural network in an initial state that is adjacent to the g lobal maximum, the resulting trajectory of the network may not converg e to the nearby maximum, but may instead go off to another maximum. At the same time, another set of recent results shows that, if an analog neural network is used to optimize the same objective function, then every local maximum of the objective function has a nontrivial domain of attraction, and conversely, the only equilibria that are attractive are the local maxima of the objective function. This raises the quest ion as to whether analog neural networks offer some advantages over bi nary neural networks for optimizing the same objective function. As a motivation for this line of inquiry, we study the problem of decoding an algebraic block code using a neural network. It is shown that the b inary neural network implementation has the undesirable property that all the global maxima of the objective function have a zero radius of attraction, In contrast, if an analog neural network is used to maximi ze exactly the same objective function, the region of attraction of ea ch maximum contains not only the associated ''orthant'' of the state s pace, but also some points not in this orthant. In other words, the an alog implementation exhibits the desired tolerance to transmission err ors, whereas the binary neural network does not have this property. Wi th this motivation, two open questions are posed that provide a progra m of research for studying the possible superiority of analog neural n etworks over binary neural networks.