A. Pakes et P. Mcguire, Stochastic algorithms, symmetric Markov perfect equilibrium, and the 'curse' of dimensionality, ECONOMETRIC, 69(5), 2001, pp. 1261-1281
This paper introduces a stochastic algorithm for computing symmetric Markov
perfect equilibria. The algorithm computes equilibrium policy and value fu
nctions, and generates a transition kernel for the (stochastic) evolution o
f the state of the system, It has two features that together imply that it
need not be subject to the curse of dimensionality. First, the integral tha
t determines continuation values is never calculated; rather it is approxim
ated by a simple average of returns from past outcomes of the algorithm, an
approximation whose computational burden is not tied to the dimension of t
he state space. Second, iterations of the algorithm update value and policy
functions at a single (rather than at all possible) points in the state sp
ace. Random draws from a distribution set by the updated policies determine
the location of the next iteration's updates. This selection only repeated
ly hits the recurrent class of points, a subset whose cardinality is not di
rectly tied to that of the state space. Numerical results for industrial or
ganization problems show that our algorithm can increase speed and decrease
memory requirements by several orders of magnitude.