Appropriate likelihood ratio tests and marginal distributions for evolutionary tree models with constraints on parameters

We show how to make appropriate likelihood ratio tests for evolutionary tre e models when parameters such as edge (intermodes or branches) lengths have nonnegativity constraints. In such cases, under the null model of an edge length being zero, the marginal distribution of this parameter is proven to be a "half-normal", that is, 50% zero values and 50% the positive half of a normal distribution. Other constrained parameters, such as the proportion of invariant sites, give similar results. To make likelihood ratio tests b etween nested models, e.g., H-0: homogeneous site rates, and H-1: site rate s follow a gamma distribution with variance 1/k, then asymptotically as seq uence length increases, the distribution under H-0, becomes a mixture of ch i(2) distributions, in this case 50% chi(0)(2), and 50% chi(1)(2) (where th e subscript denotes degrees of freedom, i.e., not the usually assumed 100% chi(1)(2); which leads to a conservative test). Such mixtures are sometimes called <(chi)over bar>(2) distributions. Simulations show that even with s equences as short as 125 sites, some parameters, including the proportion o f invariant sites, fit asymptotic distributions closely.