CONQUER: a continual query system for update monitoring in the WWW

The World Wide Web (the Web) has made an enormous amount of data freely acc essible over the Internet. However, finding the right information in the mi dst of this mountain of data has been likened to finding the proverbial nee dle in a haystack. Commonly used search engines (e.g. AltaVista) and direct ory services (e.g. Yahoo) have practical but limited success. The exponenti al growth of the Web is increasing the haystack rapidly. Instead of pull-ba sed browsing, update monitoring is a promising area of research where the s ystem brings the right information to the right user at the right time. In this paper we present the design and implementation of the CONQUER continua l query system, designed for update monitoring over the Web information sou rces. A Continual Query (CQ) is a standing query that monitors update of in terest using distributed triggers and notifies the user of changes whenever an update of interest reaches specified thresholds or some time limit is r eached. In contrast to normal queries whose scope is limited to past and pr esent data, the scope of a continual query also includes future data. The f irst contribution of the CONQUER system is the specification language and t he inherent semantics of continual queries. The second contribution consist s of the mechanisms for efficient and scalable processing of large numbers of continual queries. The third contribution is the three-tier architecture that provides active capabilities at both mediator tier and wrapper tier a nd interoperability among multiple web information sources.