A frozen 18.5 million page snapshot of part of the Web has been created to
enable and encourage meaningful and reproducible evaluation of Web search s
ystems and techniques. This collection is being used in an evaluation frame
work within the Text Retrieval Conference (TREC) and will hopefully provide
convincing answers to questions such as, "Can link information result in b
etter rankings?", "Do longer queries result in better answers?", and, "Do T
REC systems work well on Web data?" The snapshot and associated evaluation
methods are described and an invitation is extended to participate. Prelimi
nary results are presented for an effectivess comparison of six TREC system
s working on the snapshot collection against five well-known Web search sys
tems working over the current Web. These suggest that the standard of docum
ent rankings produced by public Web search engines is by no means state-of-
the-art. (C) 1999 Published by Elsevier Science B.V. All rights reserved.