On stabilization of gradient-based training strategies for computationallyintelligent systems

This paper develops a novel training methodology for computationally intell igent systems utilizing gradient information in parameter updating. The dev ised scheme uses the first-order dynamic model of the training procedure an d applies the variable structure systems approach to control the training d ynamics. This results in an optimal selection of the learning rate, which i s continually updated as prescribed by the adopted strategy. The parameter update rule is then mixed with the conventional error backpropagation metho d in a weighted average. The paper presents an analysis of the imposed dyna mics, which is the response of the training dynamics driven solely by the i nputs designed by variable structure control approach. The analysis continu es with the global stability proof of the mixed training methodology and th e restrictions on the design parameters. The simulation studies presented a re focused on the advantages of the proposed scheme with regards to the com pensation of the adverse effects of the environmental disturbances and its capability to alleviate the inherently nonlinear behavior of the system und er investigation. The performance of the scheme is compared with that of a conventional backpropagation, It is observed that the method presented is r obust under noisy observations and time varying parameters due to the integ ration of gradient descent technique with variable structure systems method ology, In the application example studied, control of a two degrees of free dom direct-drive robotic manipulator is considered. A standard fuzzy system is chosen as the controller in which the adaptation is carried out only on the defuzzifier parameters.