Skip to content

Generalizing to unseen head poses in facial expression recognition and action unit intensity estimation

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Facial expression analysis is challenged by the numerous degrees of freedom regarding head pose, identity, illumination, occlusions, and the expressions itself. It currently seems hardly possible to densely cover this enormous space with data for training a universal well-performing expression recognition system. In this paper we address the sub-challenge of generalizing to head poses that were not seen in the training data, aiming at getting along with sparse coverage of the pose subspace. For this purpose we (1) propose a novel face normalization method called FaNC that massively reduces pose-induced image variance; (2) we compare the impact of the proposed and other normalization methods on (a) action unit intensity estimation with the FERA 2017 challenge data (achieving new state of the art) and (b) facial expression recognition with the Multi-PIE dataset; and (3) we discuss the head pose distribution needed to train a pose-invariant CNNbased recognition system. The proposed FaNC method normalizes pose and facial proportions while retaining expression information and runs in less than 2 ms. When comparing results achieved by training a CNN on the output images of FaNC and other normalization methods, FaNC generalizes significantly better than others to unseen poses if they deviate more than 20° from the poses available during training. Code and data are available.
Original languageEnglish
Title of host publication2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019)
Number of pages8
ISBN (Electronic)978-1-7281-0089-0, 978-1-7281-0088-3
ISBN (Print)978-1-7281-0090-6
Publication statusPublished - 11 Jul 2019
Event14th IEEE International Conference on Automatic Face and Gesture Recognition - Lille, France
Duration: 14 May 201918 May 2019


Conference14th IEEE International Conference on Automatic Face and Gesture Recognition
Abbreviated titleFG 2019
Internet address


  • GeneralizingToUnseenHeadPoses_cameraready_FG2019

    Rights statement: © © 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

    Accepted author manuscript (Post-print), 3.99 MB, PDF document

Related information

Relations Get citation (various referencing formats)

ID: 13255159