Randomized urn models revisited using stochastic approximation

This paper presents the link between stochastic approximation and clinical trials based on randomized urn models investigated by Bai and Hu [Stochastic Process. Appl. 80 (1999) 87.101], Bai and Hu [Ann. Appl. Probab. 15 (2005) 914.940] and Bai, Hu and Shen [J. Multivariate Anal. 81 (2002) 1.18]. We reformulate the dynamics of both the urn composition and the assigned treatments as standard stochastic approximation (SA) algorithms with remainder. Then, we derive the a.s. convergence and the asymptotic normality [central limit theorem (CLT)] of the normalized procedure under less stringent assumptions by calling upon the ODE and SDE methods. As a second step, we investigate a more involved family of models, known as multi-arm clinical trials, where the urn updating depends on the past performances of the treatments. By increasing the dimension of the state vector, our SA approach provides this time a new asymptotic normality result.