Motion understanding: Task-directed attention and representations that link perception with action

This short paper outlines my position on a future direction for computation al research on visual motion understanding. The direction combines motion p erception, visual attention, action representation and computational vision . Due the breadth of literature in these areas, the paper cannot present a comprehensive review of any one topic. The review is a selective one, a sel ection that attempts to make some particular points. I claim that task-dire cted attentive processing is a largely unexplored dimension in the computat ional motion field. I recount in the context of motion understanding a past argument that in order to make vision systems general, attention is one of the components of the strategy. No matter how sophisticated the methods be come for extracting motion information from image sequences, it will not be possible to achieve the goal of human-like performance without integrating the optimization of processing that attention provides. Virtually all past surveys of computational models of motion processing completely ignore att ention. However, the concept has crept into work over the years in a variet y of ways. A second claim is that the biology of attention offers some inte resting insights to guide future development. Many computational authors ha d previously commented that too little is known about how biological vision systems use task-directed attention in motion processing; this is no longe r true. Here, I briefly summarize biological evidence that attentive proces sing affects all aspects of visual perception including motion, and again e mphasize that this paper does not do justice to the breadth and depth of th e field. New findings provide a critical link between the perception of vis ual actions and their execution. Together these findings point to a strateg y for motion understanding closely related to that presented more than two decades ago.