H-SWEB: a hierarchical scheduling system for distributed WWW server clusters

In this paper we present a model for dynamically scheduling HTTP requests a cross clusters of servers, optimizing the use of client resources as well a s the scattered server nodes. We also present a system, H-SWEB, implementin g our techniques and showing experimental improvements of over 250%, which have been achieved through utilizing a global approach to scheduling reques ts. This is the first system to provide a hierarchical scheduling mechanism for distributed HTTP server clusters incorporating dynamic client-server t ask distribution and distributed data access. H-SWEB uses sophisticated sch eduling techniques in monitoring and adapting to workload variation at the client and server clusters for supporting typical digital library tasks, su ch as fast WWW image browsing, We provide a discussion of our system archit ecture and implementation, and briefly summarize the experimental and analy tical results that have been achieved. Copyright (C) 2000 John Wiley & Sons , Ltd.