RECEIVER-OPERATED CHARACTERISTIC CURVE ANALYSIS OF 2 ALGORITHMS ASSESSING HUMAN GROWTH-HORMONE PULSATILE SECRETION (PULSAR, CLUSTER) - COMPARISON OF PEAK DETECTION EFFICACY

Computer-based peak identification algorithms reduce observer bias in the analysis of pulsatile hormone secretion. With increasing peak dete ction stringency, an algorithm will detect varying proportions of true -positive and false-positive peaks, determining its receiver-operated characteristics (ROC). To demonstrate that ROC curve analysis can char acterize algorithm performance, we analyzed growth hormone (hGH) profi les from 94 children obtained with different hGH assay techniques [rad ioimmunoassay (RIA), immunoradiometric assay (IRMA)] at different samp ling intervals (group A: 1 h/RIA; group B: 1 h/IRMA; group C: 20 min/R IA; group D: 20 min/IRMA), using the PULSAR and CLUSTER algorithms. Th e area under the ROC curve (AUC) was taken to compare the efficacy of both algorithms over a range of peak recognition stringency thresholds kept constant between algorithms, using hGH noise series for threshol d calibration and the results of multiple visual inspection as referen ce standards. AUC by PULSAR ranged from 0.926 (group C) to 0.961 (grou p A), indicating good algorithm performance. AUC by CLUSTER ranged fro m 0.869 (group B) to 0.916 (group D) in the 20-min series, decreasing to 0.756 (group C) and 0.868 (group A) in the 1-hour series. At lower sampling intensity, significant discordant sensitivity existed between algorithms for RIA (p < 0.001) and IRMA (p < 0.0026). When adjusted t o a high, assay-specific, comparable stringency, and employed on 20-mi n sampling hGH data, both the CLUSTER and PULSAR algorithm operated at a similarly high peak detection efficacy. The PULSAR algorithm appear s to be more robust when hGH series with lower sampling intensities ar e analyzed. Until objective validation techniques become generally ava ilable, we suggest that different algorithms be tested using reference data sets and ROC curve analysis to select the most efficient algorit hm and peak detection stringency threshold for the chosen assay and sa mpling conditions.