A TASK-ORIENTED APPROACH TO RETRIEVAL-SYSTEM EVALUATION

There is a need for improved methods to evaluate the effectiveness of end-user information retrieval systems. We performed a task-oriented a ssessment of two MEDLINE searching systems, one which promotes Boolean searching on human-indexed thesaurus terms and the other natural lang uage searching on words in the title, abstract, and indexing terms. Ea ch was used by medical students to answer clinical questions. Students were able to use each system successfully, with no significant differ ences in questions correctly answered, time taken, relevant articles r etrieved, or user satisfaction between the systems. This approach to e valuation was successful in measuring effectiveness of system use and demonstrates that both types of systems can be used equally well. with minimal training.