Experimentation as a way of life: Okapi at TREC

The Okapi system has been used in a series of experiments on the TREC colle ctions, investigating probabilistic models, relevance feedback and query ex pansion, and interaction issues. The TREC-6 ad hoc task was used to test an application of a new relevance weighting formula, which takes account of d ocuments judged nonrelevant. The application was to a form Of blind feedbac k (using the top-ranked documents from an initial search to improve the que ry formulation for a subsequent search, without actual relevance feedback, on the assumption that these top-ranked documents are likely to be relevant ). In the routing task, the problem is one of query optimization based on a training set with known relevant documents; investigations for TREC-6 incl uded using a form of simulated annealing for this purpose. A significant fe ature of this work is the need to avoid overfitting of the training sample. In the interactive:track, methodology remains the major problem we do not yet know how to conduct controlled laboratory experiments which provide goo d information about information retrieval interaction. The Okapi team has b een particularly interested in the relation between the functionalities ass ociated with relevance feedback and the ability of searchers to make use of these functionalities. TREC provides an excellent environment and set of t ools for,investigating automatic systems; its value for interactive systems is not yet proven. (C) 1999 Elsevier Science Ltd. All rights reserved.