The problem of selecting actions in environments that are dynamic and not c
ompletely predictable or observable is a central problem in intelligent beh
avior. In AI, this translates into the problem of designing controllers tha
t can map sequences of observations into actions so that certain goals are
achieved. Three main approaches have been used in AI for designing such con
trollers: the programming approach, where the controller is programmed by h
and in a suitable high-level procedural language, the planning approach, wh
ere the control is automatically derived from a suitable description of act
ions and goals, and the learning approach, where the control is derived fro
m a collection of experiences. The three approaches exhibit successes and l
imitations. The focus of this paper is on the planning approach. More speci
fically, we present an approach to planning based on various state models t
hat handle various types of action dynamics (deterministic and probabilisti
c) and sensor feedback (null, partial, and complete). The approach combines
high-level representations languages for describing actions, sensors, and
goals, mathematical models of sequential decisions for making precise the v
arious planning tasks and their solutions, and heuristic search algorithms
for computing those solutions. The approach is supported by a computational
tool we have developed that accepts high-level descriptions of actions, se
nsors, and goals and produces suitable controllers. We also present empiric
al results and discuss open challenges.