DUALITY BETWEEN LEARNING MACHINES - A BRIDGE BETWEEN SUPERVISED AND UNSUPERVISED LEARNING

We exhibit a duality between two perceptrons that allows us to compare the theoretical analysis of supervised and unsupervised learning task s. The first perceptron has one output and is asked to learn a classif ication of p patterns. The second (dual) perceptron has p outputs and is asked to transmit as much information as possible on a distribution of inputs. We show in particular that the maximum information that ca n be stored in the couplings for the supervised learning task is equal to the maximum information that can be transmitted by the dual percep tron.