Methods for characterizing variability and uncertainty: Comparison of bootstrap simulation and likelihood-based approaches

Variability arises due to differences in the value of a quantity among diff erent members of a population. Uncertainty arises due to lack of knowledge regarding the true value of a quantity for a given member of a population. We describe and evaluate two methods for quantifying both variability and u ncertainty. These methods, bootstrap simulation and a likelihood-based meth od, are applied to three datasets. The datasets include a synthetic sample of 19 values from a Lognormal distribution, a sample of nine values obtaine d from measurements of the PCB concentration in leafy produce, and a sample of five values for the partitioning of chromium in the flue gas desulfuriz ation system of coal-fired power plants. For each of these datasets, we emp loy the two methods to characterize uncertainty in the arithmetic mean and standard deviation, cumulative distribution functions based upon fitted par ametric distributions, the 95th percentile of variability, and the 63rd per centile of uncertainty for the 81st percentile of variability. The latter i s intended to show that it is possible to describe any point within the unc ertain frequency distribution by specifying an uncertainty percentile and a variability percentile. Using the bootstrap method, we compare results bas ed upon use of the method of matching moments and the method of maximum lik elihood for fitting distributions to data. Our results indicate that with o nly 5-19 data points as in the datasets we have evaluated, there is substan tial uncertainty based upon random sampling error. Both the boostrap and li kelihood-based approaches yield comparable uncertainty estimates in most ca ses.