Least squares and entropy: A penalty function perspective

Mathematical measures of entropy as defined by Shannon and doss entropy as defined by Kullback and Leibler are currently in vogue in the field of econ ometrics, primarily due to the comprehensive work of Golan, Judge, and Mill er. An alternative interpretation of the entropy measure as a penalty funct ion over deviations is presented. and a number of parallels are drawn with least squares estimators it is demonstrated that both approaches may be app lied to the general linear model. The causes of differences in estimated pa rameter values are described. and some suggestions for the formulation of e ntropy-based econometric problems ale presented.