MIXED MARKOV DECISION-PROCESSES IN A SEMI-MARKOV ENVIRONMENT WITH DISCOUNTED CRITERION

This paper presents a new model: the mixed Markov decision process (MD P) in a semi-Markov environment with discounted criterion. It describe s a system which behaves like a MDP except that the system is influenc ed by its semi-Markov process environment. Following each state transi tion of the environment, the MDP model changes among discrete time MDP , continuous time MDP, and semi-MDP. After presenting the model, we sh ow the validity of the optimality equation and the existence of epsilo n-optimal policies. Finally, the mixed MDP in a Markov environment is transformed into a discrete time MDP. (C) 1998 Academic Press.