Improved dynamic programming methods for optimal control of lumped-parameter stochastic systems

New dynamic programming methods are developed to solve stochastic control p roblems with a larger number of state variables than previously possible. T hese methods apply accurate interpolation to numerical approximation of con tinuous cost-to-go functions, greatly reducing the number of discrete state s that must be evaluated. By efficiently incorporating information on first and second derivatives, the approximation reduces computational effort by several orders of magnitude over traditional methods. Consequently, it is p ractical to apply dynamic programming to complex stochastic problems with a larger number of state variables than traditionally possible. Results are presented for hypothetical reservoir control problems with up to seven stat e variables and two random inputs.