Feature combination strategies for saliency-based visual attention systems

Bottom-up or saliency-based visual attention allows primates to defect nons pecific conspicuous targets in cluttered scenes. A classical metaphor, deri ved from electrophysiological and psychophysical studies, describes attenti on as a rapidly shiftable "spotlight". We use a model that reproduces the a ttentional scan paths of this spotlight. Simple multi-scale "feature maps" detect local spatial discontinuities in intensity, color, and orientation, and are combined into a unique "master" or "saliency" map. The saliency map is sequentially scanned, in order of decreasing saliency, by the focus of attention. We here study the problem of combining feature maps, from differ ent visual modalities (such as color and orientation), into a unique salien cy map. Four combination strategies are compared using three databases of n atural color images: (1) Simple normalized summation, (2) linear combinatio n with learned weights, (3) global nonlinear normalization followed by summ ation, and (4) local nonlinear competition between salient locations follow ed by summation. Performance was measured as the number of false detections before the most salient target was found. Strategy (1) always yielded poor est performance and (2) best performance, with a threefold to eightfold imp rovement in time to find a salient target. However, (2) yielded specialized systems with poor generalization. Interestingly, strategy (4) and its simp lified, computationally efficient approximation (3) yielded significantly b etter performance than (1), with up to fourfold improvement, while preservi ng generality. (C) 2001 SPIE and IS&T.