Software review of distribution fitting programs: Crystal Ball and BestFitAdd-In to @RISK

Simulation software programs continue to evolve and to meet the needs of ri sk analysts. In the past several years, two spreadsheet add-in programs add ed the capability of fitting distributions to data to their tool kits using classical statistical (i.e., non-Bayesian) methods. Crystal Ball version 4 .0 now contains this capability in its standard program (and in Crystal Bal l Pro version 4.0), while the BestFit software program is a component of th e @RISK Decision Tools Suite that can also be purchased as a stand-alone pr ogram. Both programs will automatically fit distributions using maximum lik elihood estimators to continuous data and provide goodness-of-fit statistic s based on chi-squared, Kolmogorov-Smirnov, and Anderson-Darling tests. Bes tFit will also fit discrete distributions, and for all distributions it off ers the option of optimizing the fit based on the goodness-of-fit parameter s. Analysts should be wary of placing too much emphasis on the goodness-of- fit statistics given their limitations, and the fact that only some of the statistics are appropriately corrected to account for the fact that the dis tribution parameters are also fit using the data. These programs dramatical ly simplify efforts to use maximum likelihood estimation to fit distributio ns. However, the fact that a program is used to fit distributions should no t be viewed as validation that the data have been fitted and interpreted co rrectly. Both programs rely heavily on the analyst's judgment and will allo w analysts to fit inappropriate distributions. Currently, both programs cou ld be improved by adding the ability to perform extensive basic exploratory data analysis and to give regression diagnostics that are needed to satisf y critical analysts or reviewers. Given that Bayesian methods are central t o risk analysis, adding the capability of fitting distributions by combinin g data with prior information would greatly increase the utility of these p rograms.