Optimal Control and Replacement with State-Dependent Failure Rate: An Invariant Measure Approach

Stochastic control problems in which the payoff depends on the running maximum of a diffusion process are considered. Such processes provide appealing models for physical processes that evolve in a continuous and increasing manner and fail at a random time. The controller must make two decisions: first, she must choose how fast to work (this decision determines the rate of profit as well as the proximity of failure), and second, she must decide when to replace a deteriorated system with a new one. Preventive replacement becomes an important option when the cost for replacement after a failure is larger than the cost of a preventive replacement. Single-cycle and long-term average criteria are used to evaluate the control and replacement decisions. We model the process via a martingale problem formulation. This enables the long-term average control problem to be rephrased as an LP over the invariant measures of the process. We identify the invariant measures corresponding to each control and replacement decision and determine the optimal solution using an iterative scheme.