Nonhomogeneous Markov decision processes with Borel state space - The average criterion with nonuniformly bounded rewards

This paper deals with nonhomogeneous Markov decision processes with Borel s tate space and nonuniformly bounded rewards under the average criterion. Fi rst, under the minorant-type assumption we prove the existence of an approp riate solution to the optimality equations. Second, from the optimality equ ations we also establish the existence of epsilon(greater than or equal to 0)-optimal Markov policies under the additional conditions. Third, some suf ficient conditions for the validity of the assumptions in this paper and se veral examples such as inventory/production systems are provided. Finally, as an application of the optimality equations, a rolling horizon algorithm is given.