COMBINING LINEAR DISCRIMINANT FUNCTIONS WITH NEURAL NETWORKS FOR SUPERVISED LEARNING

A novel supervised learning method is proposed by combining linear dis criminant functions with neural networks. The proposed method results in a tree-structured hybrid architecture. Due to constructive learning , the binary tree hierarchical architecture is automatically generated by a controlled growing process for a specific supervised learning ta sk. Unlike the classic decision tree, the linear discriminant function s are merely employed in the intermediate level of the tree for heuris tically partitioning a large and complicated task into several smaller and simpler subtasks in the proposed method. These subtasks are dealt with by component neural networks at the leaves of the tree according ly. For constructive learning, growing and credit-assignment algorithm s are developed to serve for the hybrid architecture. The proposed arc hitecture provides an efficient way to apply existing neural networks (e.g. multi-layered perceptron) for solving a large scale problem. We have already applied the proposed method to a universal approximation problem and several benchmark classification problems in order to eval uate its performance. Simulation results have shown that the proposed method yields better results and faster training in comparison with th e multi-layered perceptron.