Jp. Nadal et N. Parga, DUALITY BETWEEN LEARNING MACHINES - A BRIDGE BETWEEN SUPERVISED AND UNSUPERVISED LEARNING, Neural computation, 6(3), 1994, pp. 491-508
We exhibit a duality between two perceptrons that allows us to compare
the theoretical analysis of supervised and unsupervised learning task
s. The first perceptron has one output and is asked to learn a classif
ication of p patterns. The second (dual) perceptron has p outputs and
is asked to transmit as much information as possible on a distribution
of inputs. We show in particular that the maximum information that ca
n be stored in the couplings for the supervised learning task is equal
to the maximum information that can be transmitted by the dual percep
tron.