GOVERNMENT INFORMATION EXPERT-SYSTEMS - A QUANTITATIVE-EVALUATION

In this article-the first published quantitative evaluation of knowled ge-based systems (KBS) or so-called expert systems-the authors quantit atively compare and contrast two systems: POINTER and Government Docum ents Reference Aid (GDRA). In a test based on fifteen typical U.S. gov ernment document reference questions about the federal level of govern ment, POINTER answered 65 percent of the questions correctly while GDR A answered only 37 percent correctly An analysis of keystroke efficien cy revealed that POINTER required 120 strokes in the reference intervi ew and 60 for the question negotiation phase while GDRA needed 120 key strokes in the reference interview but only 45 during its question neg otiation. The discussion and implication section should help developer s of knowledge-based computer systems focus their future activities in this area and reassure human reference librarians who work with gover nment information that these systems still have a way to go before the y are truly competent systems. Nonetheless, the first generation of ex pert systems for depository libraries could already be playing a wides pread, if modest, role in assisting with federal level reference quest ions.