SPECIALIZATION PROCESSES IN ONLINE UNSUPERVISED LEARNING

From the recent analysis of supervised learning by on-line gradient de scent in multilayered neural networks it is known that the necessary p rocess of student specialization can be delayed significantly. We demo nstrate that this phenomenon also occurs in various models of unsuperv ised learning. A solvable model of competitive learning is presented, which identifies prototype vectors suitable for the representation of high-dimensional data. The specific case of two overlapping clusters o f data and a matching number of prototype vectors exhibits non-trivial behaviour like almost stationary plateau configurations. As a second example scenario we investigate the application of Sanger's algorithm for principal component analysis in the presence of two relevant direc tions in input space. Here, the fast learning of the first principal c omponent may lead to an almost complete loss of initial knowledge abou t the second one.