Optimal experimentation in a changing environment

This paper studies optimal experimentation by a monopolist who faces an unk nown demand curve subject to random changes, and who maximizes profits over an infinite horizon in continuous time. We show that there are two qualita tively very different regimes, determined by the discount rate and the inte nsities of demand curve switching, and the dependence of the optimal policy on these parameters is discontinuous. One regime is characterized by extre me experimentation and good tracking of the prevailing demand curve, the ot her by moderate experimentation and poor tracking. Moreover, in the latter regime the agent eventually becomes "trapped" into taking actions in a stri ct subset of the feasible set.