Textual data mining to support science and technology management

This paper surveys applications of data mining techniques to large text col lections, and illustrates how those techniques can be used to support the m anagement of science and technology research. Specific issues that arise re peatedly in the conduct of research management are described, and a textual data mining architecture that extends a classic paradigm for knowledge dis covery in databases is introduced. That architecture integrates information retrieval from text collections, information extraction to obtain data fro m individual texts, data warehousing for the extracted data, data mining to discover useful patterns in the data, and visualization of the resulting p atterns. At the core of this architecture is a broad view of data mining-th e process of discovering patterns in large collections of data-and that ste p is described in some detail. The final section of the paper illustrates h ow these ideas can be applied in practice, drawing upon examples from the r ecently completed first phase of the textual data mining program at the Off ice of Naval Research. The paper concludes by identifying some research dir ections that offer significant potential for improving the utility of textu al data mining for research management applications.