MEAN-FIELD THEORY FOR BATCHED TD(LAMBDA)

A representation-independent mean-field dynamics is presented for batc hed TD(lambda). The task is learning to predict the outcome of an indi rectly observed absorbing Markov process. In the case of linear repres entations, the discrete-time deterministic iteration is an affine map whose fixed point can be expressed in closed form without the assumpti on of linearly independent observation vectors. Batched linear TD(A) i s proved to converge with probability 1 for all lambda. Theory and sim ulation agree on a random walk example.