CONTRACTION CONDITIONS FOR AVERAGE AND ALPHA-DISCOUNT OPTIMALITY IN COUNTABLE STATE MARKOV GAMES WITH UNBOUNDED REWARDS

The goal of this paper is to provide a theory of N-person Markov games with unbounded cost, for a countable state space and compact action s paces. We investigate both the finite and infinite horizon problems. F or the latter, we consider the discounted cost as well as the expected average cost. We present conditions for the infinite horizon problems for which equilibrium policies exist for all players within the stati onary policies, and show that the costs in equilibrium policies exist for all players within the stationary policies, and show that the cost s in equilibrium satisfy the optimality equations. Similar results are obtained for the finite horizon costs, for which equilibrium policies are shown to exist for all players within the Markov policies. As spe cial case of N-person games, we investigate the zero-sum (2 players) g ame, for which we establish the convergence of the value iteration alg orithm. We conclude by studying an application of a zero-sum Markov ga me in a queueing model.