DENSITY-ESTIMATION ON THE STIEFEL MANIFOLD

This paper develops the theory of density estimation on the Stiefel ma nifold V-k,V-m where V-k,V-m is represented by the set of m x k matric es X such that X'X= I-k, the kxk identity matrix. The density estimati on by the method of kernels is considered. proposing two classes of ke rnel density estimators with small smoothing parameter matrices and fo r kernel functions of matrix argument. Asymptotic behavior of various statistical measures of the kernel density estimators is investigated ibr small smoothing parameter matrix and/or for large sample size. Som e decompositions of the Stiefel manifold V-k,V-m play useful roles in the investigation, and the general discussion is applied and examined for a special kernel function. Alternative methods of density estimati on are suggested, using decompositions. (C) 1998 Academic Press.