On performance potentials and conditional Monte Carlo for gradient estimation for Markov chains

We consider the problem of sample path-based gradient estimation for long-r un (steady-state) performance measures defined on discrete-time Markov chai ns. We show how two estimators - one derived using the likelihood ratio met hod with conditional Monte Carlo and splitting, and the other derived using performance potentials and perturbation analysis - are related. In particu lar, one can be expressed as the conditional expectation of a suitably weig hted average of the other. This demonstrates yet another connection between the two gradient estimation techniques of perturbation analysis and the li kelihood ratio method.