Optimal stopping and Gittins' indices for piecewise deterministic evolution processes

We study the optimal stopping problem for a class of continuous time random evolutions described by stochastic differential equations with alternating renewal processes as noise sources. The exact solution of this stopping pr oblem provides, in explicit form, an expression for the Gittins' indices ne eded to derive the optimal scheduling of a class of multi-armed bandit prob lems in continuous time. The underlying random processes to which the bandi ts' arms obey are random velocity models. Such processes are commonly used to describe, in the fluid limit, the random production flows delivered by f ailure prone machines.