Canonical correlation coefficients of high-dimensional Gaussian vectors: Finite rank case

Consider a Gaussian vector z=(x.,y.)., consisting of two sub-vectors x and y with dimensions p and q, respectively. With n independent observations of z, we study the correlation between x and y, from the perspective of the canonical correlation analysis. We investigate the high-dimensional case: both p and q are proportional to the sample size n. Denote by .uv the population cross-covariance matrix of random vectors u and v, and denote by Suv the sample counterpart. The canonical correlation coefficients between x and y are known as the square roots of the nonzero eigenvalues of the canonical correlation matrix ..1xx.xy..1yy.yx. In this paper, we focus on the case that .xy is of finite rank k, that is, there are k nonzero canonical correlation coefficients, whose squares are denoted by r1...rk>0. We study the sample counterparts of ri,i=1,.,k, that is, the largest k eigenvalues of the sample canonical correlation matrix S.1xxSxyS.1yySyx, denoted by .1....k. We show that there exists a threshold rc.(0,1), such that for each i.{1,.,k}, when ri.rc, .i converges almost surely to the right edge of the limiting spectral distribution of the sample canonical correlation matrix, denoted by d+. When ri>rc, .i possesses an almost sure limit in (d+,1], from which we can recover ri.s in turn, thus provide an estimate of the latter in the high-dimensional scenario. We also obtain the limiting distribution of .i.s under appropriate normalization. Specifically, .i possesses Gaussian type fluctuation if ri>rc, and follows Tracy.Widom distribution if ri<rc. Some applications of our results are also discussed.