USER CHOICES - A NEW YARDSTICK FOR THE EVALUATION OF RANKING ALGORITHMS FOR INTERACTIVE QUERY EXPANSION

The performance of eight ranking algorithms was evaluated with respect to their effectiveness in ranking terms for query expansion. The eval uation was conducted within an investigation of interactive query expa nsion and relevance feedback in a real operational environment. This s tudy focuses on the identification of algorithms that most effectively take cognizance of user preferences. User choices (i.e. the terms sel ected by the searchers for the query expansion search) provided the ya rdstick for the evaluation of the eight ranking algorithms. This metho dology introduces a user-oriented approach in evaluating ranking algor ithms for query expansion in contrast to the standard, system-oriented approaches. Similarities in the performance of the eight algorithms a nd the ways that these algorithms rank terms were the main focus of th is evaluation. The findings demonstrate that the r-lohi, wpq, emim, an d porter algorithms have similar performance in bringing good terms to the top of a ranked list of terms for query expansion. However, furth er evaluation of the algorithms in different (e.g. full-text) environm ents is needed before these results can be generalized beyond the cont ext of the present study.