On the validity of MAPE as a measure of population forecast accuracy

The mean absolute percent error (MAPE) is the summary measure most often us ed for evaluating the accuracy of population forecasts. While MAPE has many desirable criteria, we argue from both normative and relative standpoints that the widespread practice of exclusively using it for evaluating populat ion forecasts should be changed. Normatively, we argue that MAPE does not m eet the criterion of validity because as a summary measure it overstates th e error found in a population forecast. We base this argument on logical gr ounds and support it empirically, using a sample of population forecasts fo r counties. From a relative standpoint, we examine two alternatives to MAPE , both sharing with it, the important conceptual feature of using most of t he information about error. These alternatives are symmetrical MAPE (SMAPE) and a class of measures known as M-estimators. The empirical evaluation su ggests M-estimators do not overstate forecast error as much as either MAPE or SMAPE and are, therefore, more valid measures of accuracy. We consequent ly recommend incorporating M-estimators into the evaluation toolkit. Becaus e M-estimators do not meet the desired criterion of interpretative ease as well as MAPE, we also suggest another approach that focuses on nonlinear tr ansformations of the error distribution.