Redescending estimators for data reconciliation and parameter estimation

Gross error detection is crucial for data reconciliation and parameter esti mation, as gross errors can severely bias the estimates and the reconciled data. Robust estimators significantly reduce the effect of gross errors (or outliers) and yield less biased estimates. An important class of robust es timators are maximum likelihood estimators or M-estimators. These are commo nly of two types, Huber estimators and Hampel estimators. The former signif icantly reduces the effect of large outliers whereas the latter nullifies t heir effect. In particular, these two estimators can be evaluated through t he use of an influence function, which quantifies the effect of an observat ion on the estimated statistic. Here, the influence function must be bounde d and finite for an estimator to be robust. For the Hampel estimators the i nfluence function becomes zero for large outliers, nullifying their effect. On the other hand, Huber estimators do not reject large outliers; their in fluence function is simply bounded. As a result, we consider the three part redescending estimator of Hampel and compare its performance with a Huber estimator, the Fair function. A major advantage to redescending estimators is that it is easy to identify outliers without having to perform any explo ratory data analysis on the residuals of regression. Instead, the outliers are simply the rejected observations. In this study, the redescending estim ators are also tuned to the particular observed system data through an iter ative procedure based on the Akaike information criterion, (AIC). This appr oach is not easily afforded by the Huber estimators and this can have a sig nificant impact on the estimation. The resulting approach is incorporated w ithin an efficient non-linear programming algorithm. Finally, all of these features are demonstrated on a number of process and literature examples fo r data reconciliation. (C) 2001 Elsevier Science Ltd. All rights reserved.