3D human pose estimation in video for human-computer/robot interaction

Rongtian Huo, Qing Gao, Jing Qi, Zhaojie Ju

Research output: Chapter in Book/Report/Conference proceedingConference contribution


3D human pose estimation is widely used in motion capture, human-computer interaction, virtual character driving and other fields. The current 3D human pose estimation has been suffering from depth blurring and self-obscuring problems to be solved. This paper proposes a human pose estimation network in video based on a 2D lifting to 3D approach using transformer and graph convolutional network(GCN), which are widely used in natural language processing. We use transformer to obtain sequence features and use graph convolution to extract features between local joints to get more accurate 3D pose coordinates. In addition, we use the proposed 3D pose estimation network for animated character motion generation and robot motion following and design two systems of human-computer/robot interaction (HCI/HRI) applications. The proposed 3D human pose estimation network is tested on the Human3.6M dataset and outperforms the state-of-the-art models. Both HCI/HRI systems are designed to work quickly and accurately by the proposed 3D human pose estimation method.
Original languageEnglish
Title of host publicationIntelligent Robotics and Applications
Subtitle of host publication16th International Conference, ICIRA 2023, Hangzhou, China, July 5–7, 2023, Proceedings, Part VII
EditorsHuayong Yang, Honghai Liu, Jun Zou, Zhouping Yin, Lianqing Liu, Geng Yang, Xiaoping Ouyang, Zhiyong Wang
ISBN (Electronic)9789819964987
ISBN (Print)9789819964970
Publication statusPublished - 13 Oct 2023
EventInternational Conference on Intelligent Robotics and Applications - Hangzhou, China
Duration: 5 Jul 20237 Jul 2023

Publication series

NameLecture Notes in Computer Science
PublisherSpringer Nature
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


ConferenceInternational Conference on Intelligent Robotics and Applications


  • Human Pose Estimation
  • Human-Computer Interaction
  • Human-Robot Interaction
  • Deep Learning

Cite this