Use of general-purpose negation detection to augment concept indexing of medical documents: A quantitative study using the UMLS

dObjectives: To test the hypothesis that most instances of negated concepts in dictated medical documents can be detected by a strategy that relies on tools developed for tile parsing of formal (computer) languages-specifical ly, a lexical scanner ("lexer") that uses regular expressions to generate a finite state machine, and a parser that relies on a restricted subset of c ontext-free grammars, known as LALR(1) grammars. Methods: A diverse training set of 40 medical documents from a variety of s pecialties was manually inspected and used to develop a program (Negfinder) that contained rules to recognize a large set of negated patterns occurrin g in the text. Negfinder's lexer and parser were developed using tools norm ally used to generate programming language compilers. The input to Negfinde r consisted of medical narrative that was preprocessed to recognize UMLS co ncepts: the text of a recognized concept had been replaced with a coded rep resentation that included its UMLS concept ID. The program generated an ind ex with one entry per instance of a concept in tile document, where the pre sence or absence of negation of that concept was recorded. This information was used to mark up the text of each document by color-coding it to make i t easier to inspect. The parser was then evaluated in two ways: 1) a test s et of 60 documents (30 discharge summaries, 30 surgical notes) marked-up by Negfinder was inspected visually to quantify false-positive and false-nega tive results; and 2) 1 different test set of 10 documents was independently examined for negatives by a human observer and by Negfinder, and the resul ts were compared. Results: In the first evaluation using marked-up documents, 8,358 instances of UMLS concepts were detected in the 60 documents, of which 544 were nega tions detected by the program and verified by human observation (true-posit ive results, or TPs). Thirteen instances were wrongly flagged as negated (f alse-positive results, or FPs), and the program missed 27 instances of nega tion (false-negative results, or FNs), yielding a sensitivity of 95.3 perce nt and a specificity of 97.7 percent. hi tile second evaluation using indep endent negation detection, 1,869 concepts were detected in 10 documents, wi th 135 TPs, 12 Fps, and 6 FNs, yielding a sensitivity of 95.7 percent and a specificity of 91.8 percent. One of the words "no," "denies/denied," "not, " or "without" was present in 92.5 percent of all negations. Conclusions: Negation of most concepts in medical narrative can be reliably detected by a simple strategy. The reliability of detection depends on sev eral factors, the most important being tile accuracy of concept matching.