CONNECTIVITY AND PERFORMANCE TRADEOFFS IN THE CASCADE CORRELATION LEARNING ARCHITECTURE

The Cascade Correlation [1] is a very flexible, efficient and fast alg orithm for supervised learning. It incrementally builds the network by adding hidden units one at a time, until the desired input/output map ping is achieved. It connects all the previously installed units to th e new unit being added. Consequently, each new unit in effect adds a n ew layer and the fan-in of the hidden and output units keeps on increa sing as more units get added. The resulting structure could be hard to implement in VLSI, because the connections are irregular and the fan- in is unbounded. Moreover, the depth or the propagation delay through the resulting network is directly proportional to the number of units and can be excessive. We have modified the algorithm to generate netwo rks with restricted fan-in and small depth (propagation delay) by cont rolling the connectivity. Our results reveal that there is a tradeoff between connectivity and other performance attributes like depth, tota l number of independent parameters, learning time, etc. When the numbe r of inputs or outputs is small relative to the size of the training s et, a higher connectivity usually leads to faster learning, and fewer independent parameters, but it also results in unbounded fan-in and de pth. Strictly layered architectures with restricted connectivity, on t he other hand, need more epochs to learn and use more parameters, but generate more regular structures, with smaller, limited fan-in and sig nificantly smaller depth (propagation delay), and may be better suited for VLSI implementations. When the number of inputs or outputs is not very small compared to the size of the training set, however, a stric tly layered topology is seen to yield an overall better performance.