Information-Theoretic Measures of Fit for Univariate and Multivariate Linear Regressions

For the purpose of measuring the relative importance of independent variables in a multiple regression, Kruskal (1987) proposed an averaging procedure over all possible orderings of these variables.The present article uses this suggestion, but it is based on a different measure, from statistical information theory, and it extends the result to systems of equations.