MODEL-FREE CONTROL OF NONLINEAR STOCHASTIC-SYSTEMS WITH DISCRETE-TIMEMEASUREMENTS

Consider the problem of developing a controller for general (nonlinear and stochastic) systems where the equations governing the system are unknown. Using discrete-time measurements, this paper presents an appr oach for estimating a controller without building or assuming a model for the system (including such general models as differential/differen ce equations, neural networks, fuzzy logic rules, etc.), Such an appro ach has potential advantages in accommodating complex systems with pos sibly time-varying dynamics. Since control requires some mapping, taki ng system information, and producing control actions, the controller i s constructed through use of a function approximator (FA) such as a ne ural network or polynomial (no FA is used for the unmodeled system equ ations), Creating the controller involves the estimation of the unknow n parameters within the FA. However, since no functional form is being assumed for the system equations, the gradient of the loss function f or use in standard optimization algorithms is not available. Therefore , this paper considers the use of the simultaneous perturbation stocha stic approximation algorithm, which requires only system measurements (not a system model), Related to this, a convergence result for stocha stic approximation algorithms with time-varying objective functions an d feedback is established. It is shown that this algorithm can greatly enhance the efficiency over more standard stochastic approximation al gorithms based on finite-difference gradient approximations.