Objective verification of the SAMEX '98 ensemble forecasts

During May 1998, the Center for Analysis and Prediction of Storms (CAPS) at the University of Oklahoma coordinated a multi-institution numerical forec ast project known as the Storm and Mesoscale Ensemble Experiment (SAMEX). S AMEX involved, for the first time, the real-time operation of four differen t ensembles of mesoscale models over the same region of the United States. The main purpose of this paper is the evaluation of the ensemble forecasts, performed at a relatively coarse resolution of 30 km. An additional SAMEX goal not discussed here is to compare the value of the ensemble forecasts a gainst single forecasts made over smaller subregions of the Great Plains at both intermediate (10 km) and high (3 km) resolution. The SAMEX '98 ensembles consisted of a single 36-h control forecast from th e ARPS (at CAPS), the Penn State-NCAR fifth-generation Mesoscale Model (at NSSL), and the Eta Model and Regional Spectral Model (at NCEP), all with ho rizontal resolutions of approximately 30 km, and perturbed runs, resulting in a grand ensemble of 25 members. The forecasts of geopotential heights, t emperatures, and moisture were verified against the Eta operational analyse s, rather than observations. Unlike global ensembles, which tend to be usef ul in the medium range, the mesoscale SAMEX ensembles provided useful infor mation in the short range. A major result is that the performance of the en semble of multiple forecast systems is much better than that of each indivi dual ensemble system, probably because it represents more realistically the current uncertainties in both models and initial conditions. A similar adv antage from the use of multimodel, multianalysis systems has been observed with global ensembles. The SAMEX results also show that perturbations to mo del physics parameterizations, as well as the use of consistent perturbatio ns in the boundary conditions, are important for regional ensemble forecast ing. Efforts are now under way to compare the ensemble forecasts against th ose made using higher spatial resolution, and follow-on SAMEX experiments a re anticipated in other geographical areas and weather regimes. Although th e main results of this paper appear to be very robust, they were based on a small number of cases, and similar experiments carried out during other pe riods will help to test their significance.