APPLICATION OF ITERATIVE DYNAMIC-PROGRAMMING TO VERY HIGH-DIMENSIONALSYSTEMS

When iterative dynamic programming is used for optimal control of very high dimensional systems, we seek a piecewise linear continuous contr ol policy and use only a single grid point for the state vector. For c onvergence, a multi-pass method is used where at the beginning of each pass the initial region over which the control is taken is reduced by a factor. Convergence of such a procedure is systematic and reliable in the optimal control of a system described by 130 differential equat ions and having 130 control variables. The use of only 10 stages with piecewise linear continuous control yields excellent results. The use of 42 stages caused no computational difficulties. The attractive feat ure of the method is that the optimization can be performed easily on a personal computer, without requiring much memory.