COMPARISON OF PREDICTION METHODS WHEN ONLY A FEW COMPONENTS ARE RELEVANT

We consider prediction in a multiple regression model where we also lo ok on the explanatory variables as random. If the number of explanator y variables is large, then the common least squares multiple regressio n solution may not be the best one. We give a methodology for comparin g certain alternative prediction methods by asymptotic calculations an d perform such a comparisons for four specific methods. The results in dicate that none of these methods dominates the others, and that the d ifference between the methods typically (but not always) is small when the number of observations is large. In particular, principal compone nt regression does well when the eigenvalues corresponding to componen ts not correlated with the dependent variables (i.e., the irrelevant e igenvalues) are extremely small or extremely large. Partial least squa res regression does well for intermediate irrelevant eigenvalues. A ma ximum likelihood-type method dominates the others asymptotically, at l east in the case of one relevant component.