Markov models are useful when a decision problem involves risk that is
continuous over time, when the timing of events is important, and whe
n important events may happen more than once. Representing such clinic
al settings with conventional decision trees is difficult and may requ
ire unrealistic simplifying assumptions. Markov models assume that a p
atient is always in one of a finite number of discrete health states,
called Markov states. All events are represented as transitions from o
ne state to another. A Markov model may be evaluated by matrix algebra
, as a cohort simulation, or as a Monte Carlo simulation. A newer repr
esentation of Markov models, the Markov-cycle tree, uses a tree repres
entation of clinical events and may be evaluated either as a cohort si
mulation or as a Monte Carlo simulation. The ability of the Markov mod
el to represent repetitive events and the time dependence of both prob
abilities and utilities allows for more accurate representation of cli
nical settings that involve these issues.