This paper examines the use of "Robot Exclusion Protocol" to restrict the a
ccess of search engine robots to 10 major American university websites belo
nging to institutions recently named among "America's Most Wired" universit
ies (Gan, 1997). An analysis of web site searching and interviews with web
server administrators at these sites shows that the decision to use this pr
ocedure is largely technical and is typically made by the web server admini
strator. The implications of this decision for openness in scholarly commun
ication and for the future of academic, university-based web publishing are
discussed.