AN EFFICIENT HEURISTIC FOR A PARTIALLY OBSERVABLE MARKOV DECISION-PROCESS OF MACHINE REPLACEMENT

There is, so far, only limited practical experience applying solution schemes for real-life partially observable Markov decision processes ( POMDP's). In this work we address the special-case POMDP associated wi th the famous machine-replacement problem. The machine deteriorates do wn a series of states according to known transition probabilities. A s tate is identified by a probability of producing a defective item. Onl y a sample of the produced items is observable at each stage, in which it is to be decided whether to replace the machine or not. We suggest a very simple heuristic decision-rule that can easily handle replacem ent-type problems of large size and which is based on the Howard solut ion of the fully observable version of the problem. By a simulation ex perimental design we compare the performance of this heuristic relativ e to the generic POMDP solution algorithm which has been proposed by L ovejoy. Copyright (C) 1997 Elsevier Science Ltd