Text database discovery on the Web: Neural net based approach

As large numbers of text databases have become available on the Web, many e fforts have been made to solve the text database discovery problem: finding which text databases (out of many candidates) are most likely to provide r elevant documents to a given query. In this paper, we propose a neural net based approach to this problem. First, we present a neural net agent that l earns about underlying text databases from the user's relevance feedback. F or a given query, the neural net agent, which is sufficiently trained on th e basis of the backpropagation learning mechanism, discovers the text datab ases associated with the relevant documents and retrieves those documents e ffectively. In order to scale our approach with the large number of text da tabases, we also propose the hierarchical organization of neural net agents which reduces the total training cost at the acceptable level. Finally, we evaluate the performance of our approach by comparing it to those of the c onventional well-known statistical approaches.