Nonlinear mapping networks

Among the many dimensionality reduction techniques that have appeared in th e statistical literature, multidimensional scaling and nonlinear mapping ar e unique for their conceptual simplicity and ability to reproduce the topol ogy and structure of the data space in a faithful and unbiased manner. Howe ver, a major shortcoming of these methods is their quadratic dependence on the number of objects scaled, which imposes severe limitations on the size of data sets that can be effectively manipulated. Here we describe a novel approach that combines conventional nonlinear mapping techniques with feed- forward neural networks, and allows the processing of data sets orders of m agnitude larger than those accessible with conventional methodologies. Root ed on the principle of probability sampling, the method employs a classical algorithm to project a small random sample, and then "learns" the underlyi ng nonlinear transform using a multilayer neural network trained with the b ack-propagation algorithm. Once trained, the neural network can be used in a feed-forward manner to project the remaining members of the population as well as new, unseen samples with minimal distortion. Using examples from t he fields of image processing and combinatorial chemistry, we demonstrate t hat this method can generate projections that are virtually indistinguishab le from those derived by conventional approaches. The ability to encode the nonlinear transform in the form of a neural network makes nonlinear mappin g applicable to a wide variety of data mining applications involving very l arge data sets that are otherwise computationally intractable.