HADAMARD CONJUGATIONS AND MODELING SEQUENCE EVOLUTION WITH UNEQUAL RATES ACROSS SITES

This paper considers the many different distributions that may approxi mate the distribution of site rates in DNA sequences and shows how the Hadamard conjugation may be modified to take these into account, This is done for both 2-state and 4-state data. Distributions which give s imple closed forms include the gamma (Gamma) distribution, the inverse Gaussian distribution (which is similar to the lognormal), and a mixt ure of either of these with a proportion of sites which cannot change (invariant sites). It is seen that the tail of a distribution can have major effects upon the coefficient of variation of site rates. Becaus e the Hadamard conjugation can be used to either correct data or predi ct the data given the model (i.e., the likelihood of site patterns), l ight is shed on properties of maximum likelihood tree selection with u nequal site rates, Analysis of rRNA shows how unequal rates across sit es can change the optimal tree. Maximum likelihood analysis also shows that distinct distributions fit each data set, with the Gamma often n ot being the best. Analyzing both these data and a long stretch of pri mate mtDNA reveals evidence of many ''hidden'' multiple substitutions, while signals not corresponding to the preferred biological tree gene rally decrease as unequal rates are allowed for, Last, we discuss the expected behavior of sequences evolving by models where stabilizing se lection alone explains unequal site rates. Such models do not explain ''synapomorphies'' or informative changes in ancient molecules, becaus e while stabilizing selection can vastly decrease change at a site, it will also vastly accelerate back-substitution (leaving only a covario n model to explain old synapomorphies). When and why models allowing a continuous distribution of site rates (e.g., Gamma) will approximate covarion evolution requires further study. (C) 1997 Academic Press.