Deep correlation mining for multi-task image clustering

Xiaoqiang Yan, Kaiyuan Shi, Yangdong Ye*, Hui Yu

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

27 Downloads (Pure)


Multi-task clustering (MTC) aims to enhance the performance of each individual task by leveraging the correlation information among them. Existing MTC algorithms usually first extract the feature representations of each task and then learn the relationships among multiple tasks for clustering. However, the multi-task correlations are not embedded into the feature learning in existing MTC. In addition, many real applications, such as image clustering, always perform visual feature extraction and clustering assignment separately, which often results in local optimal clustering resolutions. In this study, an end-to-end MTC framework, named Deep correlation mining for Multi-Task image Clustering (DMTC), is proposed to explore multi-task correlations and conduct image clustering simultaneously. Specifically, DMTC consists of two sub-networks: a between-task network (B-net) and a within-task network (W-net), which learn the correlations among multiple tasks and the relationships in each individual task, respectively, based on a deep convolutional network. To optimize B-net, an optimization procedure is proposed as follows: (1) DMTC builds a pseudo-graph to discover similar samples among tasks and obtain the positive pairs of possible related tasks. (2) A discriminator is designed to calculate the mutual information between the deep and shallow representations of related tasks, which can estimate the relatedness between each pair of related tasks. After that, the trained parameters in B-net are transferred to the within-task networks (W-net) as their initialized parameters, in which the above optimization procedure is performed again to obtain the final cluster partition by end-to-end training. Experimental results on NUS-Wide, Caltech-256, Cifar-100 and Pascal VOC demonstrate that our proposed DMTC method1 compares favorably to the state-of-the-art methods.

Original languageEnglish
Article number115973
Number of pages11
JournalExpert Systems with Applications
Early online date4 Oct 2021
Publication statusPublished - 1 Jan 2022


  • Correlation mining
  • Deep clustering
  • Image clustering
  • Multi-task clustering
  • UKRI
  • EP/N025849/1


Dive into the research topics of 'Deep correlation mining for multi-task image clustering'. Together they form a unique fingerprint.

Cite this