ANALYSIS OF PROPORTIONS IN THE PRESENCE OF OVER-DISPERSION UNDER-DISPERSION

Procedures for testing homogeneity of proportions, in the presence of over-dispersion or under-dispersion, occurring in several groups in to xicology (teratology or mutagenicity) or other similar fields, are dev eloped. We consider C(alpha) (Neyman, 1959, in Probability and Statist ics: The Harold Cramer Volume, pp. 213-234, New York: Wiley) or score type tests (Rao, 1947, Proceedings of the Cambridge Philosophical Soci ety 44, 50-57) based on a parametric model, namely, the extended beta- binomial model (Prentice, 1986, Journal of the American Statistical As sociation 81, 321-327) and two semi-parametric models using quasi-like lihood (Wedderburn, 1974, Biometrika 61, 439-447) and extended quasi-l ikelihood (Nelder and Pregibon, 1987, Biometrika 74, 221-232). These p rocedures and a recent procedure by Rao and Scott (1992, Biometrics 48 , 577-585), based on the concept of design effect and effective sample size, are compared, through simulation, in terms of size, power, and robustness for departures from data distribution and dispersion homoge neity. To study robustness in terms of departure from data distributio n, we simulate data from the beta-binomial distribution, the probit no rmal binomial distribution, and the legit normal binomial distribution . Simulation shows evidence that, for litter sizes and number of litte rs that may arise in practice, a score test, based on the quasi-likeli hood, performs best in that it holds nominal level well in all data di stribution situations considered here, it shows some edge in power ove r some other statistics in some situations, and also shows robustness in presence of moderate dispersion heterogeneity. This statistic has a very simple form, and it requires estimates of the parameters only un der the null hypotheses.