Clustering based on conditional distributions in an auxiliary space

We study the problem of learning groups or categories that are local in the continuous primary space but homogeneous by the distributions of an associ ated auxiliary random variable over a discrete auxiliary space. Assuming th at variation in the auxiliary space is meaningful, categories will emphasiz e similarly meaningful aspects of the primary space. From a data set consis ting of pairs of primary and auxiliary items, the categories are learned by minimizing a Kullback-Leibler divergence-based distortion between (implici tly estimated) distributions of the auxiliary data, conditioned on the prim ary data. Still, the categories are defined in terms of the primary space. An online algorithm resembling the traditional Hebb-type competitive learni ng is introduced for learning the categories. Minimizing the distortion cri terion turns out to be equivalent to maximizing the mutual information betw een the categories and the auxiliary data. In addition, connections to dens ity estimation and to the distributional clustering paradigm are outlined. The method is demonstrated by clustering yeast gene expression data from DN A chips, with biological knowledge about the functional classes of the gene s as the auxiliary data.