Improvement of unconstrained appearance-based gaze tracking with LSTM

Guoxu Li, Lihong Dai, Qing Gao, Hongwei Gao, Zhaojie Ju

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Gaze tracking is not only an important research direction in computer vision but also an important non-verbal clue in human life. What is important is that the direction of gaze can be used as a reference for judging a person's intentions. In order to improve the accuracy of predicting gaze direction, a model of 3D gaze tracking based on bidirectional Long Short-Term Memory (LSTM) is proposed in this paper. The backbone network of the model is ResNet and its variants. The output of the model is the angular error of gaze direction. To improve the accuracy of the model prediction, the attention mechanism is adopted in this work. The ablation experiments are conducted on the selected Gaze360, which is a dataset with sufficiently large and diverse data. The angular error of the proposed model decreases from 13.5° to 12.6°.

Original languageEnglish
Title of host publication2022 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2022 - Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
Number of pages6
ISBN (Electronic)9781665452588, 9781665452571
ISBN (Print)9781665452595
Publication statusPublished - 18 Nov 2022
Event2022 IEEE International Conference on Systems, Man, and Cybernetics - Prague, Czech Republic
Duration: 9 Oct 202212 Oct 2022

Publication series

NameConference Proceedings - IEEE International Conference on Systems, Man and Cybernetics
ISSN (Print)1062-922X


Conference2022 IEEE International Conference on Systems, Man, and Cybernetics
Abbreviated titleSMC 2022
Country/TerritoryCzech Republic


  • Attention mechanism
  • gaze tracking
  • LSTM
  • ResNet

Cite this