Rigorous new approach to constructing a gold standard for validating new diagnostic criteria, as exemplified by the eosinophilia-myalgia syndrome

Background: Constructing diagnostic criteria, a common problem in clinical medicine, is particularly difficult for diseases that lack a pathognomonic "gold standard. " To develop an improved strategy for constructing such cri teria, we used the eosinophilia-myalgia syndrome as an example. The goal, f or research classifications, was to construct validated clinically sensible criteria and to develop improved methods that can be used for other disord ers. Methods: Using a "pattern-based" approach with data from several separate s ources, a committee of investigators first prepared and informally tested c riteria for the diagnosis of eosinophilia-myalgia syndrome. A gold standard challenge set of reports of cases and noncases was independently generated and separately validated by an external panel of clinical experts. The cri teria were then tested using the gold standard set, and interobserver varia bility and diagnostic accuracy were determined. Results: Interobserver variability showed the following mean proportionate agreements: 98.7% for the presence of specific criteria elements, 99% to 10 0% for diagnosis, and 97% to 98% for diagnostic pattern. K Values were corr espondingly high. Diagnostic accuracy showed sensitivity at 88%, specificit y at 97%, and overall accuracy at 92%. Conclusions: The proposed criteria are accurate and reproducible, and can b e used in future clinical investigations of the eosinophilia-myalgia syndro me. The new strategy and methods developed for this challenge can be valuab le for solving analogous problems in constructing criteria for other clinic al disorders.