VALUE OF SEARCH RESULTS AS A WHOLE AS THE BEST SINGLE MEASURE OF INFORMATION-RETRIEVAL PERFORMANCE

Value of search results as a whole, a utility measure, was found to be the best single measure of interactive information performance (succe ss) among the 20 measures selected for study (Su, 1991). The study sug gests that this measure provides a simple way for system comparison an d eliminates problems of IR evaluation with multiple measures. Value o f search results as a whole is a measure which asks for a user's ratin g on the usefulness of a set of search results based on a Likert 7-poi nt scale. This measure gives a numeric basis for comparing information retrieval performance but it does not indicate why one set of search results is rated more useful than others or how search results or syst ems can be improved to be more useful or successful. To further our un derstanding of the measure and to enhance its usefulness, the current paper explores two research questions: (1) What are the conceptual cat egories or dimensions of the: users' reasons for assigning particular ratings on the value of search results? (2) What are the relationships between these dimensions of value and the dimensions of success ident ified in the earlier study (Su, 1991)? Verbal data collected by the pr evious study will be analyzed to answer the research questions. The cu rrent paper presents results from this analysis and discusses implicat ions for theory and practice. It also compares current findings with t hose from other user criteria studies. (C) 1998 Elsevier Science Ltd. All rights reserved.