THE CASCADE-CORRELATION LEARNING - A PROJECTION PURSUIT LEARNING PERSPECTIVE

Cascade-correlation (Cascor) is a popular supervised learning architec ture that dynamically grows layers of hidden neurons of fixed nonlinea r activations (e.g., sigmoids), so that the network topology (size, de pth) can be efficiently determined. Similar to a cascade-correlation l earning network (CCLN), a projection pursuit learning network (PPLN) a lso dynamically grows the hidden neurons. Unlike a CCLN where cascaded connections from the existing hidden units to the new candidate hidde n unit are required to establish high-order nonlinearity in approximat ing the residual error, a PPLN approximates the high-order nonlinearit y by using trainable parametric or semiparametric nonlinear smooth act ivations based on minimum mean squared error criterion. An analysis is provided to show that the maximum correlation training criterion used in a CCLN tends to produce hidden units that saturate and thus makes it more suitable for classification tasks instead of regression tasks as evidenced in the simulation results. It is also observed that this critical weakness in CCLN can also potentially carry over to classific ation tasks, such as the two-spiral benchmark used in the original CCL N paper.