ASYMPTOTIC ANALYSIS OF STOCHASTIC GRADIENT-BASED ADAPTIVE FILTERING ALGORITHMS WITH GENERAL COST-FUNCTIONS

This paper presents an analysis of stochastic gradient-based adaptive algorithms with general cost functions. The analysis holds under mild assumptions on the inputs and the cost function. The method of analysi s is based on an asymptotic analysis of fixed stepsize adaptive algori thms and gives almost sure results regarding the behavior of the param eter estimates, whereas previous stochastic analyses typically conside r mean and mean square behavior. The parameter estimates are shown to enter a small neighborhood about the optimum value and remain there fo r a finite length of time. Furthermore, almost sure exponential bounds are given for the rate of convergence of the parameter estimates. The asymptotic distribution of the parameter estimates is shown to be Gau ssian with mean equal to the optimum value and covariance matrix that depends on the input statistics. Specific adaptive algorithms that fal l under the framework of this paper are signed error least mean squre (LMS), dual sign LMS, quantized state LMS, least mean fourth, dead zon e algorithms, momentum algorithms, and leaky LMS.