ON RELATIVE CONVERGENCE PROPERTIES OF PRINCIPAL COMPONENT ANALYSIS ALGORITHMS

We investigate the convergence properties of two different stochastic approximation algorithms for principal component analysis, and analyti cally explain some commonly observed experimental results. In our anal ysis, we use the theory of stochastic approximation, and in particular the results of Fabian, to explore the asymptotic mean square errors ( AMSE's) of the algorithms. This study reveals the conditions under whi ch the algorithms produce smaller AMSE's, and also the conditions unde r which one algorithm has a smaller AMSE than the other. Experimental study with multidimensional Gaussian data corroborate our analytical f indings. We next explore the convergence rates of the two algorithms. Our experiments and an analytical explanation reveals the conditions u nder which the algorithms converge faster to the solution, and also th e conditions under which one algorithm converges faster than the other . Finally, we observe that although one algorithm has a larger computa tion in each iteration, it leads to a smaller AMSE and converges faste r for the minor eigenvectors when compared to the other algorithm.