Error bounds for occupation measure of singularly perturbed Markov chains including transient states

Motivated by many applications in production planning, system reliability, queueing networks, and wireless communication, this work is devoted to sing ularly perturbed Markov chains with finite states. Focusing on nonstationar y processes with the inclusion of transient states, asymptotic error bounds of a sequence of suitably scaled occupation measures are derived. The main tools used include martingales and differential equations. The results are useful for analyzing structural properties of the underlying Markov chains and for designing nearly optimal and hierarchical controls of large-scale and complex systems.