To understand visual perception we must examine visual processes that opera
te over time scales longer than a single fixation. In this paper, we argue
that it is necessary to consider behavioral context in order to address thi
s issue. We consider two types of behavior that might require information a
cquired in different fixations. First, we describe behavior in the everyday
task of making a sandwich. Fixation patterns and eye-hand coordination sug
gest that memory for spatial context is fundamental for coordinated movemen
ts. We then examine a block-copying task that reveals the use of precise sp
atial memory information in saccadic targeting. In a second block-copying t
ask we show that observers are sensitive to regularities in the structure o
f the environment and that this is reflected in fixation behavior. Thus the
accrual of spatial structure of a scene across fixations appears to be imp
ortant for mediating visually guided movements. This is in contrast to the
commonly accepted idea that visual scene representations are sparse and abs
tract.