A CASE-STUDY OF THE USE OF STATISTICAL-MODELS IN FORECAST VERIFICATION - PRECIPITATION PROBABILITY FORECASTS

The traditional approach to forecast verification consists of computin g one, or at most very few, quantities from a set of forecasts and ver ifying observations. However, this approach necessarily discards a lar ge portion of the information regarding forecast quality that is conta ined in a set of forecasts and observations. Theoretically sound alter native verification approaches exist, bur these often involve computat ion and examination of many quantities in order to obtain a complete d escription of forecast quality and, thus, pose difficulties in interpr etation. This paper proposes and illustrates an intermediate approach to forecast verification, in which the multifaceted nature of forecast quality is recognized but the description of forecast quality is enca psulated in a much smaller number of parameters. These parameters are derived from statistical models fit to verification datasets. Forecast ing performance as characterized by the statistical models can then be assessed in a relatively complete manner. In addition, the fitted sta tistical models provide a mechanism for smoothing sampling variations in particular finite samples of forecasts and observations. This appro ach to forecast verification is illustrated by evaluating and comparin g selected samples of probability of precipitation (PoP) forecasts and the matching binary observations. A linear regression model is fit to the conditional distributions of the observations given the forecasts and a beta distribution is fit to the frequencies of use of the allow able probabilities. Taken together, these two models describe the join t distribution of forecasts and observations, and reduce a 21-dimensio nal verification problem to 4 dimensions (two parameters each for the regression and beta models). Performance of the selected PoP forecasts is evaluated and compared across forecast type, location, and lead ti me in terms of these four parameters (and simple functions of the para meters), and selected graphical displays are explored as a means of ob taining relatively transparent views of forecasting performance within this approach to verification.