On terminating Markov decision processes with a risk-averse objective function

We consider a class of terminating Markov decision processes with an expone ntial risk-averse objective function and compact constraint sets. We assume the existence of an absorbing cost-free terminal state Omega. positive tra nsition costs, and continuity of the transition probability and cost functi ons. Without discounting future costs in the argument of the exponential ut ility function, we establish(i) the existence of a real-valued optimal cost function which can be achieved by a stationary policy and (ii) the converg ence of value iteration and policy iteration to the unique solution of Bell man's equation. We illustrate the results with two computational examples. (C) 2001 Elsevier Science Ltd. All rights reserved.