Towards a better understanding of Web resources and server responses for improved caching

This work focuses on characterizing information about Web resources and ser ver responses that is relevant to Web caching. The approach is to study a s et of URLs at a variety of sites and gather statistics about the rate and n ature of changes compared with the resource type. In addition, we gather re sponse header information reported by the servers with each retrieved resou rce. Results from the work indicate that there is potential to reuse more c ached resources than is currently being realized due to inaccurate and none xistent cache directives. In terms of implications for caching, the relatio nships between resources used to compose a page must be considered. Embedde d images are often reused, even in pages that change frequently. This resul t both points to the need to cache such images and to discard them when the y are no longer included as part of any page. Finally, while the results sh ow that HTML resources frequently change, these changes can be in a predict able and localized manner. Separating out the dynamic portions of a page in to their own resources allows relatively static portions to be cached, whil e retrieval of the dynamic resources can trigger retrieval of new resources along with any invalidation of already cached resources. (C) 1999 Publishe d by Elsevier Science B.V. All rights reserved.