GENERATIVE LEARNING STRUCTURES AND PROCESSES FOR GENERALIZED CONNECTIONIST NETWORKS

Massively parallel networks of relatively simple computing elements of fer an attractive and versatile framework for exploring a variety of l earning structures and processes for intelligent systems. This paper b riefly summarizes some popular learning structures and processes used in such networks. It outlines a range of potentially more powerful alt ernatives for pattern-directed inductive learning in such systems. It motivates and develops a class of new learning algorithms for massivel y parallel networks of simple computing elements. We call this class o f learning processes generative for they offer a set of mechanisms for constructive and adaptive determination of the network architecture-t he number of processing elements and the connectivity among them-as a function of experience. Generative learning algorithms attempt to over come some of the limitations of some approaches to learning in network s that rely on modification of weights on the links within an otherwis e fixed network topology, for example, rather slow learning and the ne ed for an a priori choice of network architecture. Several alternative designs as well as a range of control structures and processes that c an be used to regulate the form and content of internal representation s learned by such networks are examined. Empirical results from the st udy of some generative learning algorithms are briefly summarized, and several extensions and refinements of such algorithms and directions for future research are outlined.