H. Osman et Mm. Fahmy, ON THE DISCRIMINATORY POWER OF ADAPTIVE FEEDFORWARD LAYERED NETWORKS, IEEE transactions on pattern analysis and machine intelligence, 16(8), 1994, pp. 837-842
This correspondence expands the available theoretical framework that e
stablishes a link between discriminant analysis and adaptive feed-forw
ard layered linear-output networks used as mean-square classifiers. Th
is has the advantages of providing more theoretical justification for
the use of these nets in pattern classification and gaining a better i
nsight into their behavior and about their use. We prove that, under r
easonable assumptions, minimizing the mean-square error at the network
output is equivalent to minimizing the following: 1) the difference b
etween the optimum value of a familiar discriminant criterion and the
value of this criterion evaluated in the space spanned by the outputs
of the final hidden layer, and 2) the difference between the values of
the same discriminant criterion evaluated in desired-output and actua
l-output subspaces. We also illustrate, under specific constraints, ho
w to solve the following problem: given a feature extraction criterion
, how the target coding scheme can be selected such that this criterio
n is maximized at the output of the network final hidden layer. Other
properties for these networks are explored.