An overview of the simultaneous perturbation method for efficient optimization

Multivariate stochastic optimization plays a major role in the analysis and control of many engineering systems. In almost all real-world optimization problems, it is necessary to use a mathematical algorithm that iteratively seeks out the solution because an analytical (closed-form) solution is rar ely available. In this spirit, the "simultaneous perturbation stochastic ap proximation (SPSA)" method for difficult multivariate optimization problems has been developed. SPSA has recently attracted considerable international attention in areas such as statistical. parameter estimation, feedback con trol, simulation-based optimization, signal and image processing, and exper imental. design. The essential feature of SPSA-which accounts for its power and relative ease of implementation-is the underlying gradient approximati on that requires only two measurements of the objective function regardless of the dimension of the optimization problem. This feature allows for a si gnificant decrease in the cost of optimization, especially in problems with a large number of variables to be optimized.