SELF-ORGANIZING ALGORITHMS FOR GENERALIZED EIGEN-DECOMPOSITION

We discuss a new approach to self-organization that leads to novel ada ptive algorithms for generalized eigendecomposition and its variance [ such as linear discriminant analysis (LDA)] for a single-layer linear feedforward neural network, First, we derive two novel iterative algor ithms for LDA and generalized eigen-decomposition by utilizing a const rained least-mean-squared classification error cost function, and the framework of a two-layer linear heteroassociative network performing a one-of-m classification, By using the concept of deflation, we are ab le to find sequential versions of these algorithms which extract the L DA components and generalized eigenvectors in a decreasing order of si gnificance. Second, two new adaptive algorithms are described to compu te the principal generalized eigenvectors of two matrices (as well as LDA) from two sequences of random matrices, Although iterative algorit hms for LDA exist in the literature, we give a rigorous convergence an alysis of our adaptive algorithms by using stochastic approximation th eory, and prove that our algorithms converge with probability one. As an example, we consider the problem of online interference cancellatio n in digital mobile communications, Numerical simulations are presente d demonstrating the rapid convergence of the adaptive algorithms, and their relative convergence rates.