Text classification with self-organizing maps: Some lessons learned

The self-organizing map has already found appreciation for document classif ication in the information retrieval community. The map display is a highly effective and intuitive metaphor for orientation in the information space established by a document collection. In this paper we discuss ways for usi ng self-organizing maps for document classification. Furthermore, we argue in favor of paying more attention to the fact that document collections len d themselves naturally to a hierarchical structure defined by the subject m atter of the documents. We take advantage of this fact by using a hierarchi cally organized neural network, built up from a number of independent self- organizing maps in order to enable the true establishment of a document tax onomy. As a highly convenient side effect of using such an architecture, th e time needed for training is reduced substantially and the user is provide d with an even more intuitive metaphor for visualization. Since the single layers of self-organizing maps represent different aspects of the document collection at different levels of detail, the neural network shows the docu ment collection in a form comparable to an atlas where the user may easily select the most appropriate degree of granularity depending on the actual f ocus of interest during the exploration of the document collection. (C) 199 8 Elsevier Science B.V. All rights reserved.