Distilling ordinal relation and dark knowledge for facial age estimation

Qilu Zhao, Junyu Dong, Hui Yu, Sheng Chen

Research output: Contribution to journalArticlepeer-review

145 Downloads (Pure)


In this paper, we propose a knowledge distillation approach with two teachers for facial age estimation. Due to the nonstationary patterns of facial aging process, the relative order of age labels provides more reliable information than exact age values for facial age estimation. Thus the first teacher is a novel ranking method capturing the ordinal relation among age labels. Specifically, it formulates the ordinal relation learning as a task of recovering the original ordered sequences from shuffled ones. The second teacher adopts a same model as the student that treats facial age estimation as a multi-class classification task. The proposed method leverages the intermediate representations learned by the first teacher and the softened outputs of the second teacher as supervisory signals to improve the training procedure and final performance of the compact student for facial age estimation. Hence, the proposed knowledge distillation approach is capable of distillating the ordinal knowledge from the ranking model and the dark knowledge from the multi-class classification model into a compact student, which facilitates the implementation of facial age estimation on platforms with limited memory and computation resources, such as mobile and embedded devices. Extensive experiments involving several famous datasets for age estimation have demonstrated the superior performance of our proposed method over several existing state-of-the-art methods.
Original languageEnglish
Number of pages14
JournalIEEE Transactions on Neural Networks and Learning Systems
Early online date17 Aug 2020
Publication statusEarly online - 17 Aug 2020


  • facial age estimation
  • self-supervised learning
  • jigsaw puzzles solver
  • permission prediction
  • knowledge distillation
  • dark knowledge
  • feature transfer


Dive into the research topics of 'Distilling ordinal relation and dark knowledge for facial age estimation'. Together they form a unique fingerprint.

Cite this