Deep correlation mining for multi-task image clustering

Multi-task clustering (MTC) aims to enhance the performance of each individual task by leveraging the correlation information among them. Existing MTC algorithms usually first extract the feature representations of each task and then learn the relationships among multiple tasks for clustering. However, the multi-task correlations are not embedded into the feature learning in existing MTC. In addition, many real applications, such as image clustering, always perform visual feature extraction and clustering assignment separately, which often results in local optimal clustering resolutions. In this study, an end-to-end MTC framework, named Deep correlation mining for Multi-Task image Clustering (DMTC), is proposed to explore multi-task correlations and conduct image clustering simultaneously. Specifically, DMTC consists of two sub-networks: a between-task network (B-net) and a within-task network (W-net), which learn the correlations among multiple tasks and the relationships in each individual task, respectively, based on a deep convolutional network. To optimize B-net, an optimization procedure is proposed as follows: (1) DMTC builds a pseudo-graph to discover similar samples among tasks and obtain the positive pairs of possible related tasks. (2) A discriminator is designed to calculate the mutual information between the deep and shallow representations of related tasks, which can estimate the relatedness between each pair of related tasks. After that, the trained parameters in B-net are transferred to the within-task networks (W-net) as their initialized parameters, in which the above optimization procedure is performed again to obtain the final cluster partition by end-to-end training. Experimental results on NUS-Wide, Caltech-256, Cifar-100 and Pascal VOC demonstrate that our proposed DMTC method¹ compares favorably to the state-of-the-art methods.