BOUNDS ON APPROXIMATE STEEPEST DESCENT FOR LIKELIHOOD MAXIMIZATION INEXPONENTIAL-FAMILIES

An approximate steepest descent strategy converging, in families of re gular exponential densities, to maximum likelihood estimates of densit y functions is described. These density estimates are also obtained by an application of the principle of minimum relative entropy subject t o empirical constraints. We prove tight bounds on the increase of the log-likelihood at each iteration of our strategy for families of expon ential densities whose log-densities are spanned by a set of bounded b asis functions.