PARALLEL IMPLEMENTATION OF BACKPROPAGATION NEURAL NETWORKS ON A HETEROGENEOUS ARRAY OF TRANSPUTERS

This paper analyzes parallel implementation of the backpropagation tra ining algorithm on a heterogeneous transputer network (i.e., transpute rs of different speed and memory) connected in a pipelined ring topolo gy. Training-set parallelism is employed as the parallelizing paradigm for the backpropagation algorithm. It is shown through analysis that finding the optimal allocation of the training patterns amongst the pr ocessors to minimize the time for a training epoch is a mixed integer programming problem. Using mixed integer programming optimal pattern a llocations for heterogeneous processor networks having a mixture of T8 05-20 (20 MHz) and T805-25 (25 MHz) transputers are theoretically foun d out for two benchmark problems. The time for an epoch corresponding to the optimal pattern allocations is then obtained experimentally for the benchmark problems from the 805-20, T805-25 heterogeneous network s. A Monte Carlo simulation study is carried out to statistically veri fy the optimality of the epoch time obtained from the mixed integer pr ogramming based allocations. In this study pattern allocations are ran domly generated and the corresponding time for ap epoch is experimenta lly obtained from the heterogeneous network. The mean and standard dev iation for the epoch times from the random allocations are then compar ed with the optimal epoch time. The results show the optimal epoch tim e to be always lower than the mean epoch times by more than three stan dard deviations (3 sigma) for all the sample sizes used in the study t hus giving validity to the theoretical analysis.