Initial evaluation of a continuous speech recognition program for radiology

The aims of this work were to measure the accuracy of one continuous speech recognition product and dependence on the speaker's gender and status as a native or nonnative English speaker, and evaluate the product's potential for routine use in transcribing radiology reports. IBM MedSpeak/Radiology s oftware, version 1.1 was evaluated by 6 speakers. Two were nonnative Englis h speakers, and 3 were men. Each speaker dictated a set of 12 reports. The reports included neurologic and body imaging examinations performed with 6 different modalities. The dictated and original report texts were compared, and error rates for overall, significant, and subtle significant errors we re computed. Error rate dependence on modality, native English speaker stat us, and gender were evaluated by performing t tests. The overall error rate was 10.3 +/- 3.3%. No difference in accuracy between men and women was fou nd; however, significant differences were seen for overall and significant errors when comparing native and nonnative English speakers (P = .009 and P = .008, respectively). The speech recognition software is approximately 90 % accurate, end while practical implementation issues (rather than accuracy ) currently limit routine use of this product throughout a radiology practi ce, application in niche areas such as the emergency room currently is bein g pursued. This methodology provides a convenient way to compare the initia l accuracy of different speech recognition products, and changes in accurac y over time, in a detailed end sensitive manner. Copyright (C) 2001 by W.B. Saunders Company.