A comparison between human and machine labelling of image regions

In previous work (Campbell et al, 1997 Pattern Recognition 30 555-563) a vi sion system was developed which is capable of classifying objects in outdoo r scenes. The approach involves segmenting the image into regions, obtainin g a feature-based description of each region, and then passing this descrip tion on to an artificial neural network (ANN) which has been trained to lab el the region with one of eleven possible object types. The question addres sed here is: how important is each of these features to overall performance , both in human and machine vision? A set of experiments was conducted in which human subjects were trained in the same labelling task as the ANN. The stimuli, each depicting a single im age region, were generated from a large database of urban and rural images. The subjects were then tested on both intact and degraded stimuli. The res ults suggest that certain features are particularly influential in mediatin g overall labelling performance. An equivalent experiment was carried out with the ANN. A method is presente d which allows individual features to be corrupted in such a way as to simu late the loss of certain forms of visual information. The results, which ar e broadly similar to those found in the previous experiment, imply that the ANN can provide a useful model of human image region labelling. It is anti cipated that the methodology, which draws on both computational and psychop hysical techniques, will be of use to other areas of investigation.