Jk. Peterson, THE USE OF PONTRYAGIN ESTIMATORS FOR ONLINE OPTIMAL-CONTROL SEQUENCE ESTIMATION - THE TRUCK BACKER-UPPER CASE-STUDY, Mathematical and computer modelling, 21(1-2), 1995, pp. 31-51
The Pontryagin optimality principle can be used in conjunction with on
-line (or real-time) measurements of state data to build a local model
of the control law. In this paper, we discuss and refine the use of t
his technique in the context of the simple truck backer-upper problem.
We first compare the use of feedforward, associative and CMAC neural
architectures for the local control model encoding. Algorithm implemen
tation is then done using the CMAC architecture because of its speed o
f learning and local scoping. We build temporal difference state predi
ction models for the truck dynamics and then use these predictions to
build an estimate of the best control action to take. This control act
ion is constructed from a depth first tree search used in conjunction
with optimal control information obtained by solving locally scoped co
ntrol problems via the Pontryagin optimality principle. The state to c
ontrol model can then be encoded into a variety of function approximat
ion models.