Semi-automated entry of clinical temporal-abstraction knowledge

Objectives: The authors discuss the usability of an automated tool that sup ports entry, by clinical experts, of the knowledge necessary for forming hi gh-level concerts and patterns from raw time-oriented clinical data. Design: Based on their previous work on the RESUME: system for forming high -level concerts from raw time-oriented clinical data, the authors designed a graphical knowledge acquisition (KA) tool that acquires the knowledge req uired by RESUME. This tool was designed using Protege, a general framework and set of tools for the construction of knowledge-based systems. The usabi lity of the KA tool was evaluated by three expert physicians and three know ledge engineers in three domains-the monitoring of children's growth, the c are of patients with diabetes, and protocol-based care in oncology and in e xperimental therapy for AIDS. The study evaluated the usability of the KA t ool for the entry of previously elicited knowledge. Measurements: The authors recorded the time required to understand the meth odology and the KA tool and to enter the knowledge; they examined the subje cts' qualitative comments; and they compared the output abstractions with b enchmark abstractions computed from the same data and a version of the same knowledge entered manually by RESUME: experts. Results: Understanding RESUME required 6 to 20 hours (median, 15 to 20 hour s); learning to use the KA tool required 2 to 6 hours (median, 3 to 4 hours ). Entry times for physicians varied by domain-2 to 20 hours for growth mon itoring (median, 3 hours), 6 and 12 hours for diabetes care, and 5 to 60 ho urs for protocol-based care (median, 10 hours). An increase in speed of up to 25 times (median, 3 times) was demonstrated for all participants when th e KA process was repeated. On their first attempt at using the tool to ente r the knowledge, the knowledge engineers recorded entry times similar to th ose of the expert physicians' second attempt at entering the same knowledge . Ln all cases RESUME, using knowledge entered by means of the KA tool, gen erated abstractions that were almost identical to those generated using the same knowledge entered manually. Conclusion: The authors demonstrate that the KA tool is usable and effectiv e for expert physicians and knowledge engineers to enter clinical temporal abstraction knowledge and that the resulting knowledge bases are as valid a s those produced by manual entry.