MINIMUM COMPLEXITY REGRESSION ESTIMATION WITH WEAKLY DEPENDENT OBSERVATIONS

The minimum complexity regression estimation framework, due to Barron, is a general data-driven methodology for estimating a regression func tion from a given list of parametric models using independent and iden tically distributed (i.i.d.) observations. We extend Barren's regressi on estimation framework to m-dependent observations and to strongly mi xing observations, In particular, we propose abstract minimum complexi ty regression estimators for dependent observations, which may be adap ted to a particular list of parametric models, and establish upper bou nds on the statistical risks of the proposed estimators in terms of ce rtain deterministic indices of resolvability. Assuming that the regres sion function satisfies a certain Fourier-transform-type representatio n, we examine minimum complexity regression estimators adapted to a li st of parametric models based on neural networks and, by using the upp er bounds for the abstract estimators, we establish rates of convergen ce for the statistical risks of these estimators, Also, as a key tool, we extend the classical Bernstein inequality from i.i.d. random varia bles to m-dependent processes and to strongly mixing processes.