2ND-ORDER APPROXIMATION OF THE ENTROPY IN NONLINEAR LEAST-SQUARES ESTIMATION

Measures of variability of the least-squares estimator theta are essen tial to assess the quality of the estimation. In nonlinear regression, an accurate approximation of the covariance matrix of theta is diffic ult to obtain [4]. In this paper, a second-order approximation of the entropy of the distribution of theta is proposed, which is only slight ly more complicated than the widely used bias approximation of Box [3] . It is based on the ''flat'' or ''saddle-point approximation'' of the density of theta. The neglected terms are of order O(sigma4), while t he classical first order approximation neglects terms of order O(sigma 2). Various illustrative examples are presented, including the use of the approximate entropy as a criterion for experimental design.