EVALUATION OF INFORMATION-SEEKING PERFORMANCE IN HYPERMEDIA DIGITAL LIBRARIES

Nowadays, we are witnessing the development of new information-seeking environments and applications such as hypermedia digital libraries. I nformation Retrieval (IR) is increasingly embedded in these environmen ts and plays a cornerstone role. However, in hypermedia digital librar ies IR is a part of a large and complex user-centred information-seeki ng environment. In particular, information seeking is also possible us ing non-analytical, opportunistic and intuitive browsing strategies. T his paper discusses the particular evaluation problems posed by these current developments. Current methods based on Recall (R) and Precisio n (P) for evaluating IR are discussed, and their suitability for evalu ating the performance of hypermedia digital libraries is examined. We argue that these evaluation methods cannot be directly applied, mainly because they do not measure the effectiveness of browsing strategies; the underlying notion of relevance ignores the highly interconnected nature of hypermedia information and misses the reality of how informa tion seekers work in these environments. Therefore, we propose a new q uantitative evaluation methodology, based on the structural analysis o f hypermedia networks and the navigational and search state patterns o f information seekers. Although the proposed methodology retains some of the characteristics (and criticisms) of R and P evaluations, it cou ld be more suitable than them for measuring the performance of informa tion-seeking environments where information seekers can utilize arbitr ary mixtures of browsing and query-based searching strategies. (C) 199 8 Elsevier Science B.V.