A probability and decision-model analysis of PROVOST seasonal multi-model ensemble integrations

A probabilistic analysis is made of seasonal ensemble integrations from the PROVOST project (PRediction Of climate Variations On Seasonal to interannu al Time-scales), with emphasis on the Brier score and related Murphy decomp osition, and the relative operating characteristic. To illustrate the signi ficance of these results to potential users, results from the analysis of t he relative operating characteristic are input to a simple decision model. The decision-model analysis is used to define a user-specific objective mea sure of the economic value of seasonal forecasts. The analysis is made for two simple meteorological forecast conditions or 'events', E, based on 850 hPa temperature. The ensemble integrations result from integrating four dif ferent models over the period 1979-93. For each model a set of 9-member ens embles is generated by running from consecutive analyses. Results from the Brier skill score analysis taken over all northern hemisph ere grid points indicate that, whilst the skill of individual-model ensembl es is only marginally higher than a probabilistic forecast of climatologica l frequencies, the multi-model ensemble is substantially more skilful than climatology. Both reliability and resolution are better for the multi-model ensemble than for the individual-model ensembles. This improvement arises both from the use of different models in the ensemble, and from the enhance d ensemble size obtained by combining individual-model ensembles; the latte r reason was found to be the more important. Brier skill scores are higher for years in which there were moderate or strong Fl Nino Southern Oscillati on (ENSO) events. Over Europe, only the multi-model ensembles showed skill over climatology. Similar conclusions are reached from an analysis of the r elative operating characteristic. Results from the decision-model analysis show that the economic value of se asonal forecasts is strongly dependent on the cost, C, to the user of takin g precautionary action against E, in relation to the potential loss, L, if precautionary action is not taken and E occurs. However, based on the multi -model ensemble data, the economic value can be as much as 50% of the value of a hypothetical perfect deterministic forecast. For the hemisphere as a whole, value is enhanced by restriction to ENSO years. It is shown that the re is potential economic value in seasonal forecasts for European users. Ho wever, the impact of ENSO on economic value over Europe is mixed; value is enhanced by El Nino only for some potential users with specific C/L. The techniques developed are applicable to complex E for arbitrary regions. Hence these techniques are proposed as the basis of an objective probabili stic and decision-model evaluation of operational seasonal ensemble forecas ts.