Simulation-based optimization with stochastic approximation using common random numbers

The method of Common Random Numbers is a technique used to reduce the varia nce of difference estimates in simulation optimization problems. These diff erences are commonly used to estimate gradients of objective functions as p art of the process of determining optimal values for parameters of a simula ted system. asymptotic results exist which show that using the Common Rando m Numbers method in the iterative Finite Difference Stochastic Approximatio n optimization algorithm (FDSA) can increase the optimal rate of convergenc e of the algorithm from the typical rate of k(-1/3) to the faster k(-1/2), where k is the algorithm's iteration number. Simultaneous Perturbation Stoc hastic Approximation (SPSA) is a newer and often much more efficient optimi zation algorithm, and we will show that this algorithm, too, converges fast er when the Common Random Numbers method is used. We will also provide mult ivariate asymptotic covariance matrices for both the SPSA and FDSA errors.