The World Wide Web has become an invaluable information resource but the ex
plosion of available information has made Web search a time consuming and c
omplex process. The large number of information sources and their different
levels of accessibility, reliability and associated costs present a comple
x information gathering control problem. This paper describes the rationale
, architecture, and implementation of a next generation information gatheri
ng system-a system that integrates several areas of Artificial Intelligence
research under a single umbrella. Our solution to the information explosio
n is an information gathering agent, BIG, that plans to gather information
to support a decision process, reasons about the resource trade-offs of dif
ferent possible gathering approaches, extracts information from both unstru
ctured and structured documents, and uses the extracted information to refi
ne its search and processing activities. (C) 2000 Elsevier Science B.V. All
rights reserved.