We pursue in this paper our study of approximations of values and E-sa
ddle-point policies in dynamic zero-sum games. After extending the gen
eral theorem for approximation, we study zero-sum stochastic games wit
h countable state space and unbounded immediate reward. We focus on th
e expected average payoff criterion. We use some tools developed in [M
. M. Tidball and E. Altman, SIAM J. Control Optim., 34 (1996), pp. 311
-328] to obtain the convergence of the values as well as the convergen
ce of the epsilon saddle-point policies in various approximation probl
ems. We consider several schemes of truncation of the state space (e.g
., finite state approximation) and approximations of games with discou
nt factor close to one for the game with expected average cost. We use
the extension of the general theorem for approximation to study appro
ximations in stochastic games with complete information. Finally, we c
onsider the problem of approximating the sets of policies. We obtain s
ome general results that we apply to a pursuit evasion differential ga
me.