Adaptive control of constrained finite Markov chains

An adaptive control algorithm is presented for constrained finite controlle d Markov chains with unknown transition probabilities. A finite set of alge braic constraints has been considered. The Lagrange multipliers approach is used to solve this constrained optimization problem. This scheme is such t hat at each time n estimates the control policy on the basis on Bush-Mostel ler scheme which is related to stochastic approximation procedures. We pres ent the asymptotic properties (convergence and order of convergence rate) o f the algorithm. They follow from the law of dependent large numbers, marti ngales theory and Lyapunov function analysis approaches. (C) 1999 Elsevier Science Ltd. All rights reserved.