ENERGY FUNCTIONS FOR MINIMIZING MISCLASSIFICATION ERROR WITH MINIMUM-COMPLEXITY NETWORKS

For automatic target recognition, a neural network is desired that min imizes the number of misclassifications with the minimum network compl exity. Minimizing network complexity is important for both improving g eneralization and simplifying implementation. The least mean squares ( LMS) energy function used in standard back propagation does not always produce such a network. Therefore, two minimum misclassification erro r (MME) energy functions are advanced to achieve this. Examples are gi ven in which LMS requires five times as many hidden units in a multila yer perceptron to achieve test set classification accuracy similar to that achieved with the MME functions. Examples are given to provide in sight into the nature of the LMS performance, namely that LMS approxim ates the a posteriori probabilities and class boundaries emerge indire ctly from this process. The examples also show that the MME functions tend to find local minima less often than LMS does for the same number of hidden units. This is believed to be due to the difference in netw ork complexity needed to accurately approximate a posteriori probabili ties versus class boundaries.