Assessing the temporal variation of macroeconomic forecasts by a panel of changing composition

This paper calls attention to the problem of changing panel composition in surveys of forecasters and documents the problem in the Survey of Professional Forecasters. To study the temporal variation of forecasts, we recommend analysis of the time series of predictions made by individual forecasters. This makes transparent the heterogeneity of the panel and avoids improper inferences due to changing panel composition. In the absence of knowledge of the process determining panel composition, we warn against the traditional practice of aggregate time series analysis, which conflates changes in the expectations of individual forecasters with changes in the composition of the panel. Should analysis of aggregated predictions be thought desirable as a simplifying device, we recommend analyses of sub-panels of fixed composition.