The Unifying Role of Iterative Generalized Least Squares in Statistical Algorithms

This expository paper deals with the role of iterative generalized least squares as an algorithm for the computation of statistical estimators. Relationships between various algorithms, such as Newton-Raphson, Gauss-Newton, and scoring, are studied. A parallel is made between statistical properties of the model and the structure of the numerical algorithm employed to find parameter estimates. In particular a general linearizability property that extends the concept of link function in generalized linear models is considered and its computational meaning is discussed. Maximum quasilikelihood estimators are reinterpreted so that they may exist even when there is no quasilikelihood function.