The value iteration method for countable state Markov decision processes

This paper deals with Markov decision processes with a countable state spac e. We demonstrate that a single, relatively simple condition suffices to gu arantee that the value-iteration method converges and that an optimal polic y can be computed via this method, once the existence of a solution to the average cost optimality equation has been established via any of the many a vailable sets of existence conditions. (C) 1999 Elsevier Science B.V. All r ights reserved.