Search engines and indices were created to help people find informatio
n amongst the rapidly increasing number of World Wide Web (WWW) pages.
The search engines automatically visit and index pages so that they c
an return good matches for their users' queries. The way that this ind
exing is done varies from engine to engine and the detail is usually s
ecret although the strategy is sometimes made public in general terms.
The search engines' aim is to return relevant pages quickly. On the o
ther hand, the author of a Web page has a vested interest in it rating
highly, for appropriate queries, on as many search engines as possibl
e. Some authors have an interest in their page rating well for a great
many types of query indeed - spamming has come to the Web. We treat m
odelling the workings of WWW search engines as an inductive inference
problem. A training set of data is collected, being pages returned in
response to typical, queries. Decision trees are used as the model cla
ss for the search engines' selection criteria although this is not to
say that search engines actually contain decision trees. A machine lea
rning program is used to infer a decision tree for each search engine,
an information-theory criterion being used to direct the inference an
d to prevent over-fitting. (C) 1998 Published by Elsevier Science B.V.
All rights reserved.