LEARNING ACTION PROBABILITIES FROM DELAYED REINFORCEMENT

A reinforcement scheme for learning automata, applicable to real situa tions where the reinforcement received from the environment is delayed , is presented. The scheme divides the state space into regions follow ing the boxes approach of Michie and Chambers. Each region maintains e stimates of the reward characteristics of the environment and contains a local automaton that updates action probabilities whenever the syst em state enters it. Estimates of reward characteristics are obtained u sing reinforcement received during the period of eligibility. Results obtained through computer simulation of the inverted pendulum problem are compared with the adaptive critic learning developed by Barto et a l. (1983).