Generalizing to unseen head poses in facial expression recognition and action unit intensity estimation

Philipp Werner, Frerk Saxen, Ayoub Al-Hamadi, Hui Yu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

128 Downloads (Pure)

Abstract

Facial expression analysis is challenged by the numerous degrees of freedom regarding head pose, identity, illumination, occlusions, and the expressions itself. It currently seems hardly possible to densely cover this enormous space with data for training a universal well-performing expression recognition system. In this paper we address the sub-challenge of generalizing to head poses that were not seen in the training data, aiming at getting along with sparse coverage of the pose subspace. For this purpose we (1) propose a novel face normalization method called FaNC that massively reduces pose-induced image variance; (2) we compare the impact of the proposed and other normalization methods on (a) action unit intensity estimation with the FERA 2017 challenge data (achieving new state of the art) and (b) facial expression recognition with the Multi-PIE dataset; and (3) we discuss the head pose distribution needed to train a pose-invariant CNNbased recognition system. The proposed FaNC method normalizes pose and facial proportions while retaining expression information and runs in less than 2 ms. When comparing results achieved by training a CNN on the output images of FaNC and other normalization methods, FaNC generalizes significantly better than others to unseen poses if they deviate more than 20° from the poses available during training. Code and data are available.
Original languageEnglish
Title of host publication2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019)
PublisherIEEE
Number of pages8
ISBN (Electronic)978-1-7281-0089-0, 978-1-7281-0088-3
ISBN (Print)978-1-7281-0090-6
DOIs
Publication statusPublished - 11 Jul 2019
Event14th IEEE International Conference on Automatic Face and Gesture Recognition - Lille, France
Duration: 14 May 201918 May 2019
http://fg2019.org/

Conference

Conference14th IEEE International Conference on Automatic Face and Gesture Recognition
Abbreviated titleFG 2019
CountryFrance
CityLille
Period14/05/1918/05/19
Internet address

Fingerprint

Dive into the research topics of 'Generalizing to unseen head poses in facial expression recognition and action unit intensity estimation'. Together they form a unique fingerprint.

Cite this