INFERENCE FOR NONRANDOM SAMPLES

Observational data are often analysed as if they had resulted from a c ontrolled study, and yet the tacit assumption of randomness can be cru cial for the validity of inference. We take some simple statistical mo dels and supplement them by adding a parameter theta which reflects th e degree of non-randomness in the sample. For a randomized study theta is known to be 0. We examine the profile log-likelihood for theta and the sensitivity of inference to small nonzero values of theta. Partic ular models cover the analysis of survey data with item nonresponse, t he paired comparison t-test and two group comparisons using observatio nal data with covariates. Some practical examples are discussed. Allow ing for sampling bias increases the uncertainty of estimation and weak ens the significance of treatment effects, sometimes substantially so.