Sequentially deciding between two experiments for estimating a common success probability

To estimate a success probability p, two experiments are available: individ ual Bernoulli (p) trials or the product of r individual Bernoulli (p) trial s. This problem has its roots in reliability where either single components can be tested or a system of r identical components can be rested. A total of N experiments can be performed, and the problem is to sequentially sele ct some combination (allocation) of these two experiments, along with an es timator of p, to achieve low mean squared error (MSE) of the final estimate . This scenario is similar to that of the better-known group testing proble m, but here the goal is to estimate failure rates rather than to identify d efective units. The problem also arises in epidemiological applications suc h as estimating disease prevalence. Information maximization considerations , and analysis of the asymptotic MSE of several estimators, lead to the fol lowing adaptive procedure: use the maximum likelihood estimator (MLE) to es timate p, and if this estimator is below (above) the cutpoint a(r), then ob serve an individual (product) trial at the next stage. In a Bayesian settin g with squared error estimation loss and suitable regularity conditions on the prior distribution, this adaptive procedure-replacing the MLE with the Bayes estimator-will be asymptotically Bayes. Exact computational evaluatio ns of the adaptive procedure for fixed sample sizes show that it behaves ro ughly as the asymptotics predict. The exact analyses also show parameter re gions for which the adaptive procedure achieves negative regret, as well as regions for which it achieves normalized MSE: superior to that asymptotica lly possible. An example and a discussion of extensions conclude the work.