Interobserver variability in applying a radiographic definition for ARDS

Context: Acute lung injury (ALI) and ARDS are currently defined by the Amer ican-European Consensus Conference (IIE-CC) definition criteria, which cont ain a radiographic criterion. The accuracy or reliability of this consensus radiographic definition has not been evaluated, and no radiographic defini tion of ALI-ARDS has been evaluated by a large international group of exper ts. Objective: To study the interobserver variability in applying the AECC radi ographic criterion for ALI-IIRDS. Design: Survey. Participants: A convenience sample of 21 experts selected from participants attending the 1997 Toronto Mechanical Ventilation Workshop and from member s of the National Institutes of Health ARDS Network. Outcome measures: Participants reviewed 28 randomly selected chest radiogra ph from critically ill, hypoxemic (Pao(2)/fraction of inspired oxygen ratio , < 300) patients and decided whether the radiograph fulfilled the AECC def inition for ALI-IIRDS. Results: Interobserver agreement in applying the AECC definition for ALI-AR DS was moderate (kappa = 0.55; 95% confidence interval, 0.52 to 0.57), Thir teen radiographs (43%) showed nearly complete agreement (defined as 20 or 2 1 readers in agreement). Nine radiographs (32%) had more than or equal to f ive dissenting readers. The percentage of radiographs interpreted as consis tent with Al;I-ARDS by individual readers ranged from 36 to 71%. Participan ts commented that mild infiltrates, pleural effusions, atelectasis, isolate d lower lobe involvement, radiographic technique, and overlying monitoring equipment posed the most difficulties. Conclusions: The radiographic criterion used in the current AECC definition for ALI-ARDS showed high interobserver variability when applied by expert investigators in the fields of mechanical ventilation and ARDS. This variab ility may result in differences in ALI-ARDS populations at different clinic al research centers and may make it difficult for clinicians to apply the r esults of clinical trials to their patients. Modifications to the radiograp hic criterion or annotated reference radiograph may improve the reliability of future definitions for ALI-ARDS.