Evaluation of LabRespond, a new automated validation system for clinical laboratory test results

Background: Manual validation of laboratory test results is time-consuming, creating a demand for expert systems to automate this process. We have sta rted to set up the program "LabRespond", which covers five validation level s: administrative, technical, sample, patient, and clinical validation. We present the evaluation of a prototype of an automated patient validation sy stem based on statistical methods, in contrast to the commercially availabl e program "VALAB", a rule-based automated validation system. Methods: In the present study, 163 willfully altered, erroneous test result s out of 5421 were submitted for validation to LabRespond, VALAB, and to a group of clinical chemists (n = 9) Who validated these test results manuall y. The test results rejected by three or more clinical chemists (n = 281) s erved as a secondary reference standard. Results: The error recovery rates of clinical chemists ranged from 23.9% to 71.2%. The recovery rates of LabRespond and VALAB were 77.9% and 71.8%, re spectively (difference not significant). The false-positive rates were 82.7 % for LabRespond, 83.6% for VALAB, and 27.8-86.7% for clinical chemists. Us ing the consensus of three or more clinical chemists as the secondary refer ence standard, we found error recovery rates of 64.8% for LabRespond and 72 .2% for VALAB (P = 0.06). Compared with VALAB, LabRespond detected more (P = 0.003) erroneous test results of the type that were changed from abnormal to normal. Conclusions: The statistical plausibility check used by LabRespond offers a promising automated validation method with a higher error recovery rate th an the clinical chemists participating in this study, and a performance com parable to VALAB. (C) 2000 American Association for Clinical Chemistry.