Interactive query expansion: A user-based evaluation in a relevance feedback environment

A user-centered investigation of interactive query expansion within the con text of a relevance feedback system is presented in this article. Data were collected from 25 searches using the INSPEC database. The data collection mechanisms included questionnaires, transaction logs, and relevance evaluat ions. The results discuss issues that relate to query expansion, retrieval effectiveness, the correspondence of the on-line-to-off-line relevance judg ments, and the selection of terms for query expansion by users (interactive query expansion). The main conclusions drawn from the results of the study are that: (1) one-third of the terms presented to users in a list of candi date terms for query expansion was identified by the users as potentially u seful for query expansion. (2) These terms were mainly judged as either var iant expressions (synonyms) or alternative (related) terms to the initial q uery terms. However, a substantial portion of the selected terms were ident ified as representing new ideas. (3) The relationships identified between t he five best terms selected by the users for query expansion and the initia l query terms were that: (a) 34% of the query expansion terms have no relat ionship or other type of correspondence with a query term; (b) 66% of the r emaining query expansion terms have a relationship to the query terms. Thes e relationships were: narrower term (46%), broader term (3%), related term (17%). (4) The results provide evidence for the effectiveness of interactiv e query expansion. The initial search produced on average three highly rele vant documents; the query expansion search produced on average nine further highly relevant documents. The conclusions highlight the need for more res earch on: interactive query expansion, the comparative evaluation of automa tic vs. interactive query expansion, the study of weighted Web-based or Web -accessible retrieval systems in operational environments, and for user stu dies in searching ranked retrieval systems in general.