SIMULATION STUDY OF A PANEL OF RELIABILITY INDICATORS APPLIED TO PAIRED MEASUREMENTS

Reliability is a subject of continuing discussion in biomedial special ty areas, including physical anthropology and nutritional epidemiology . The purpose of this study was to explore techniques of detecting dif ferences between two evaluators or methods. A field study in which ant hropometric dimensions would be taken by two independent evaluators on each participant in a study group was simulated. A panel of reliabili ty indicators was applied across a broad range of parameters using sim ulation, and then the panel was applied to field anthropometric data. The panel consisted of the intraclass correlation coefficient (ICC), p aired t-test, a simultaneous test of evaluator means and variances, te chnical error of measurement, mean absolute difference, and mean diffe rence. The simultaneous test for equal evaluator means and variances u ses regression to model paired differences versus paired sums. The sim ulation demonstrated general properties of the reliability indicators across many conditions of population variance, measurer bias, and meas urer error variance. High values of ICC often exist in cases in which the measurers are different. The simultaneous test is thus a powerful method for detecting measurer differences, especially when combined wi th the paired t-test. However, a single reliability indicator that is sufficient to determine all measurer inconsistencies was not identifie d. The field study and the simulation permitted the development of a l ogical approach to determining the source and magnitude of measurer di fferences using the panel of reliability indicators. (C) 1994 Wiley-Li ss, Inc.