On universal oracle inequalities related to high-dimensional linear models

This paper deals with recovering an unknown vector . from the noisy data Y.=.A..+..., where A is a known (m...n)-matrix and . is a white Gaussian noise. It is assumed that n is large and A may be severely ill-posed. Therefore, in order to estimate ., a spectral regularization method is used, and our goal is to choose its regularization parameter with the help of the data Y. For spectral regularization methods related to the so-called ordered smoothers [see Kneip Ann. Statist. 22 (1994) 835.866], we propose new penalties in the principle of empirical risk minimization. The heuristical idea behind these penalties is related to balancing excess risks. Based on this approach, we derive a sharp oracle inequality controlling the mean square risks of data-driven spectral regularization methods.