A feature fusion network for skeleton-based gesture recognition

Xiaowen You, Qing Gao, Hongwei Gao, Zhaojie Ju

Research output: Chapter in Book/Report/Conference proceedingConference contribution

14 Downloads (Pure)

Abstract

With the development of the times, the requirements for human-computer interaction methods have gradually increased, and naturalness and comfort are constantly pursued on the basis of traditional precision. Gesture is one of the innate ways of human communication, which is highly intuitive and can be employed as an effective means of natural human-computer interaction. In this paper, dynamic gestures are investigated based on the 3D skeletal information of gestures, and different cropping boxes are placed at generate global and local datasets respectively according to whether they depend on the motion trajectory of the gesture. By analyzing the geometric features of skeletal sequences, a dual-stream 3D CNN (Double_C3D) framework is proposed for fusion at the feature level, which relies on 3D heat map video streams and uses the video streams as the input to the network. Finally, the Double_C3D framework was evaluated on the SHREC dynamic gesture recognition dataset and the JHMBD dynamic behavior recognition dataset with an accuracy of 91.72% and 70.54%, respectively.
Original languageEnglish
Title of host publicationIntelligent Robotics and Applications 16th International Conference, ICIRA 2023, Hangzhou, China, July 5–7, 2023, Proceedings, Part II
EditorsHuayong Yang, Honghai Liu, Jun Zou, Zhouping Yin, Lianqing Liu, Geng Yang, Xiaoping Ouyang, Zhiyong Wang
PublisherSpringer
Pages67-78
Number of pages12
ISBN (Electronic)9789819964864
ISBN (Print)9789819964857
DOIs
Publication statusPublished - 10 Oct 2023
EventInternational Conference on Intelligent Robotics and Applications - Hangzhou, China
Duration: 5 Jul 20237 Jul 2023

Publication series

NameLecture Notes in Computer Science
PublisherSpringer
Volume14268
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349
NameLecture Notes in Artificial Intelligence
PublisherSpringer
ISSN (Print)2945-9133
ISSN (Electronic)2945-9141

Conference

ConferenceInternational Conference on Intelligent Robotics and Applications
Country/TerritoryChina
CityHangzhou
Period5/07/237/07/23

Keywords

  • 3D skeleton
  • pseudo heat map
  • 3D CNN

Fingerprint

Dive into the research topics of 'A feature fusion network for skeleton-based gesture recognition'. Together they form a unique fingerprint.

Cite this