ON THE DISCRIMINATORY POWER OF ADAPTIVE FEEDFORWARD LAYERED NETWORKS

This correspondence expands the available theoretical framework that e stablishes a link between discriminant analysis and adaptive feed-forw ard layered linear-output networks used as mean-square classifiers. Th is has the advantages of providing more theoretical justification for the use of these nets in pattern classification and gaining a better i nsight into their behavior and about their use. We prove that, under r easonable assumptions, minimizing the mean-square error at the network output is equivalent to minimizing the following: 1) the difference b etween the optimum value of a familiar discriminant criterion and the value of this criterion evaluated in the space spanned by the outputs of the final hidden layer, and 2) the difference between the values of the same discriminant criterion evaluated in desired-output and actua l-output subspaces. We also illustrate, under specific constraints, ho w to solve the following problem: given a feature extraction criterion , how the target coding scheme can be selected such that this criterio n is maximized at the output of the network final hidden layer. Other properties for these networks are explored.