EVALUATION LONG-TERM MAINTENANCE OF A LARGE MEDICAL KNOWLEDGE-BASE

Objective: Evaluate the effects of long-term maintenance activities on existing portions of a large internal medicine knowledge base. Design : Five physicians who were not among the original developers of the kn owledge base independently updated a total of 15 QMR disease profiles; each updated submission was modified by a review group serving as the ''gold standard,'' and the pre- and post-study versions of each updat ed disease profile were compared. Measurements: Numbers and types of c hanges, defined as any difference between the original version and the final version of a disease profile; reason for each change; and bibli ographic references cited by the physicians as supporting evidence. Re sults: A total of 16% of all entries were modified by the updating pro cess; up to 95% of the entries in a disease profile were affected. The two most common modifications were changes td the frequency of an ent ry, and creation of a new entry. Laboratory findings were affected muc h more often than were history, symptom, or physical exam findings. Th e dominant reason for changes was appearance of new evidence in the me dical literature. The literature cited ranged from 1944 to the present . Conclusions: This study provides an evaluation of the rate of change within the QMR medical knowledge base due to long-term maintenance. T he results show that this is a demanding activity that may profoundly affect certain portions of a knowledge base, and that different types of knowledge (e.g., simple laboratory vs expensive or invasive laborat ory findings) are affected by the process in different ways.