We introduce a notion of pathwise optimality for stochastic control problem
s over an infinite time horizon, and give sufficient conditions for the exi
stence of pathwise optimal controls. We analyze both diffusion processes an
d processes with discrete state space.