Continuity of optimal values and solutions for control of Markov chains with constraints

Citation
Mm. Tidball et al., Continuity of optimal values and solutions for control of Markov chains with constraints, SIAM J CON, 38(4), 2000, pp. 1204-1222
Citations number
46
Categorie Soggetti
Mathematics,"Engineering Mathematics
Journal title
SIAM JOURNAL ON CONTROL AND OPTIMIZATION
ISSN journal
03630129 → ACNP
Volume
38
Issue
4
Year of publication
2000
Pages
1204 - 1222
Database
ISI
SICI code
0363-0129(20000524)38:4<1204:COOVAS>2.0.ZU;2-D
Abstract
We consider in this paper constrained Markov decision processes. This type of control model has many applications in telecommunications and other fiel ds [E. Altman and A. Shwartz, IEEE Trans. Automat. Control, 34 (1989), pp. 1089-1102, E. A. Feinberg and M. I. Reiman, Probab. Engrg. Inform. Sci., 8 (1994), pp. 463-489, A. Hordijk and F. Spieksma, Adv. in Appl. Probab., 21 (1989), pp. 409-431, A. Lazar, IEEE Trans. Automat. Control, 28 (1983), pp. 1001-1007,. Nain and K. W. Ross, IEEE Trans. Automat. Control, 31 (1986), pp. 883-888, K. W. Ross and B. Chen, IEEE Trans. Automat. Control, 33 (1988 ), pp. 261-267]. We address the issue of the convergence of the value and o ptimal policies of the problem with discounted costs, to the ones for the p roblem with expected average cost. We consider the general multichain ergod ic structure. We present two stability results in this paper. We establish the continuity of optimal values and solutions of as well as some type of r obustness of some suboptimal solutions in the discount factor. Our proof re lies on same general theory on continuity of values and solutions in convex optimization that relies on well-known notions of Gamma-convergence.