SPREAD: scalable platform for reliable and efficient automated distribution

We introduce SPREAD - a new architecture for distributing and maintaining u p-to-date Web content that simultaneously employs three different mechanism s: client validation, server invalidation, and replication. Proxies within SPREAD self-configure themselves to form scalable distribution hierarchies that connect the origin servers of content providers to clients. Each proxy autonomously decides on the best mechanism based on the object's popularit y and modification rates. Requests and subscriptions propagate from edge pr oxies to the origin server through a chain of intermediate proxies. Invalid ations and replications travel in the opposite direction. SPREAD's network of proxies automatically reconfigures when proxies go down or come up, or w hen new ones are added. The ability to spontaneously form hierarchies is ba sed on a modified transparent proxying mechanism, called translucent proxyi ng, that sanitizes transparent proxying. It allows proxies to be placed in an ad-hoc fashion anywhere in the network - not just at focal points within the network that are guaranteed to see all the packets of a TCP connection . In this paper we (1) describe the architecture of SPREAD, (2) discuss how proxies determine which mechanism to use based on local observations, and (3) use a trace-driven simulation to test SPREAD's behavior in a realistic setting. (C) 2000 Published by Elsevier Science B.V. All rights reserved.