Subset-selection and ensemble methods for wavelet de-noising

Many nonparametric regression procedures are based on "subset selection": t hey choose a subset of carriers from a large or even infinite set, and then determine the coefficients of the chosen carriers by least squares. Proced ures which can be cast in this framework include Projection Pursuit, Turbo, Mars, and Matching Pursuit. Recently, considerable attention has been give n to "ensemble estimators" which combine least squares estimates obtained f rom multiple subsets of carriers. In the parametric regression setting, suc h ensemble estimators have been shown to improve on the accuracy of subset selection procedures in some situations. In this paper we compare subset se lection estimators and ensemble estimators in the context of wavelet de-noi sing. We present simulation results demonstrating that a certain class of e nsemble wavelet estimators, based on the concept of "cycle spinning", are s ignificantly more accurate than subset selection methods. These advantages hold even when the subset selection procedures can rely on an oracle to sel ect the optimal number of carriers. We compute ideal thresholds for transla tion invariant wavelet shrinkage and investigate other approaches to ensemb le wavelet estimation.