CONFIDENCE-INTERVAL PROCEDURES FOR MONTE-CARLO TRANSPORT SIMULATIONS

The problem of obtaining valid confidence intervals based on estimates from sampled distributions using Monte Carlo particle transport simul ation codes such as MCNP is examined. Such intervals can cover the tru e parameter of interest at a lower than nominal rate if the sampled di stribution is extremely right-skewed by large tallies. Modifications t o the standard theory of confidence intervals are discussed and compar ed with some existing heuristics, including batched means normality te sts. Two new types of diagnostics are introduced to assess whether the conditions of central limit theorem-type results are satisfied: The r elative variance of the variance determines whether the sample size is sufficiently large, and estimators of the slope of the right tail of the distribution are used to indicate the number of moments that exist . A simulation study is conducted to quantify the relationship between various diagnostics and coverage rates and to find sample-based quant ities useful in indicating when intervals are expected to be valid. Si mulated tally distributions are chosen to emulate behavior seen in dif ficult particle transport problems. Measures of variation in the sampl e variance s(2) are found to be much more effective than existing meth ods in predicting when coverage will be near nominal rates. Batched me ans tests are found to be overly conservative in this regard. A simple but pathological MCNP problem is presented as an example of ''false'' convergence using existing heuristics. The new methods readily detect the false convergence and show that the results of the problem, which are a factor of 4 too small, should not be used. Recommendations are made for applying these techniques in practice, using the statistical output currently produced by MCNP.