Multi-armed bandits in discrete and continuous time

We analyze Gittins' Markovian model, as generalized by Varaiya, Walrand and Buyukkoc, in discrete and continuous time. The approach resembles Weber's modification of Whittle's, within the framework, of both multiparameter pro cesses and excursion theory. It is shown that index-priority strategies are optimal, in concert with all the special cases that have been treated prev iously.