EVALUATION OF NARRATIVE TEXT FOR CASE-FINDING - THE NEED FOR ACCURACYMEASUREMENT

This article reviews the analysis of a narrative text electronic searc h technique being used in the insurance industry. We reviewed a previo usly published study of motor vehicle crashes in roadway construction workzones as well as additional data supplied by the authors with resp ect to the methods of keyword selection. The narrative text search tec hnique was evaluated with decision statistics and was found to have a sensitivity of 92.3%, 95% confidence interval 67.5-99.6%. This range o f sensitivity, at its most extreme value, led to a 32.5% underestimati on of claims prevalence. Furthermore, because the electronic search de veloped two classification categories from a limited text field (appro ximately 20 words), only half of the cases had at least one classifica tion. Systematic error estimates were used to obtain true population p roportions for crash characteristics, revealing significant underestim ations in costs. This analysis highlights the need for investigators t o apply decision statistics to narrative text searching techniques whe n they are used essentially as diagnostic test procedures on insurance claims datasets. (C) 1998 Wiley-Liss, Inc.