Unsupervised segmentation of color-texture regions in images and video

A new method for unsupervised segmentation of color-texture regions in imag es and video is presented. This method, which we refer to as JSEG, consists of two independent steps: color quantization and spatial segmentation. In the first step, colors in the image are quantized to several representative classes that can be used to differentiate regions in the image. The image pixels are then replaced by their corresponding color class labels, thus fo rming a class-map of the image. The focus of this work is on spatial segmen tation, where a criterion for "good" segmentation using the class-map is pr oposed. Applying the criterion to local windows in the class-map results in the "J-image," in which high and low values correspond to possible boundar ies and interiors of color-texture regions. A region growing method is then used to segment the image based on the multiscale J-images. A similar appr oach is applied to video sequences. An additional region tracking scheme is embedded into the region growing process to achieve consistent segmentatio n and tracking results, even for scenes with nonrigid object motion. Experi ments show the robustness of the JSEG algorithm on real images and video.