Skip to content

Generalizing to unseen head poses in facial expression recognition and action unit intensity estimation

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Facial expression analysis is challenged by the numerous degrees of freedom regarding head pose, identity, illumination, occlusions, and the expressions itself. It currently seems hardly possible to densely cover this enormous space with data for training a universal well-performing expression recognition system. In this paper we address the sub-challenge of generalizing to head poses that were not seen in the training data, aiming at getting along with sparse coverage of the pose subspace. For this purpose we (1) propose a novel face normalization method called FaNC that massively reduces pose-induced image variance; (2) we compare the impact of the proposed and other normalization methods on (a) action unit intensity estimation with the FERA 2017 challenge data (achieving new state of the art) and (b) facial expression recognition with the Multi-PIE dataset; and (3) we discuss the head pose distribution needed to train a pose-invariant CNNbased recognition system. The proposed FaNC method normalizes pose and facial proportions while retaining expression information and runs in less than 2 ms. When comparing results achieved by training a CNN on the output images of FaNC and
other normalization methods, FaNC generalizes significantly better than others to unseen poses if they deviate more than 20° from the poses available during training. Code and data are available.
Original languageEnglish
Title of host publicationThe 14th IEEE International Conference on Automatic Face and Gesture Recognition
Publication statusAccepted for publication - 22 Jan 2019
Event14th IEEE International Conference on Automatic Face and Gesture Recognition - Lille, France
Duration: 14 May 201918 May 2019


Conference14th IEEE International Conference on Automatic Face and Gesture Recognition
Abbreviated titleFG 2019
Internet address


  • GeneralizingToUnseenHeadPoses_cameraready_FG2019

    Rights statement: The embargo end date of 2050 is a temporary measure until we know the publication date. Once we know the publication date the full text of this article will be able to view shortly afterwards.

    Accepted author manuscript (Post-print), 3 MB, PDF-document

    Due to publisher’s copyright restrictions, this document is not freely available to download from this website until: 1/01/50

Related information

Relations Get citation (various referencing formats)

ID: 13255159