In this work we develop a new multivariate technique to produce regressions
with interpretable coefficients that are close to and of the same signs as
the pairwise regression coefficients. Using a multiobjective approach to i
ncorporate multiple and pairwise regressions into one objective we reduce t
his technique to an eigenproblem that represents a hybrid between regressio
n and principal component analyses. We show that our approach corresponds t
o a specific scheme of ridge regression with a total matrix added to the ma
trix of correlations.