Region based parallel hierarchy convolutional neural network for automatic facial nerve paralysis evaluation

Xin Liu, Yifan Xia, Hui Yu*, Junyu Dong, Muwei Jian, Tuan Pham

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

354 Downloads (Pure)


In this paper, we propose a parallel hierarchy convolutional neural network (PHCNN) combining a Long Short-Term Memory (LSTM) network structure to quantitatively assess the grading of facial nerve paralysis (FNP) by considering the region-based asymmetric facial features and temporal variation of the image sequences. FNP, such as Bell’s palsy, is the most common facial symptom of neuromotor dysfunctions. It causes the weakness of facial muscles for the normal emotional expression and movements. The subjective judgement by clinicians completely depends on individual experience, which may not lead to a uniform evaluation. Existing computer-aided methods mainly rely on some complicated imaging equipment, which is complicated and expensive for facial functional rehabilitation. Compared with the subjective judgment and complex imaging processing, the objective and intelligent measurement can potentially avoid this issue. Considering dynamic variation in both global and regional facial areas, the proposed hierarchical network with LSTM structure can effectively improve the diagnostic accuracy and extract paralysis detail from the low-level shape, contour to sematic level features. By segmenting the facial area into two palsy regions, the proposed method can discriminate FNP from normal face accurately and significantly reduce the effect caused by age wrinkles and unrepresentative organs with shape and position variations on feature learning. Experiment on the YouTube Facial Palsy Database and Extended CohnKanade Database shows that the proposed method is superior to the state of the art deep learning methods.
Original languageEnglish
JournalIEEE Transactions on Neural Systems and Rehabilitation Engineering
Early online date3 Sept 2020
Publication statusEarly online - 3 Sept 2020


  • RCUK
  • EP/N025849/1


Dive into the research topics of 'Region based parallel hierarchy convolutional neural network for automatic facial nerve paralysis evaluation'. Together they form a unique fingerprint.

Cite this