Skeleton-based multi-feature sharing real-time action recognition network for human-robot interaction

Zhiwen Deng, Qing Gao, Xiang Yu, Zhaojie Ju, Junkang Chen

Research output: Chapter in Book/Report/Conference proceedingConference contribution

31 Downloads (Pure)


Human-Robot Interaction (HRI) is one of the directions that deserves to be studied, and it is used in various fields (e.g., emergency rescue, telemedicine, and astronaut assistance). Action-based HRI has been proven practical in many application scenarios (e.g., industrial teleoperation and telemedicine). Among these many applications, speed and accuracy are two common optimization goals for researchers. A skeleton-based multi-feature sharing real-time action recognition network (MSR-Net) has been proposed to ensure its high accuracy and speed. Three pairs of feature inputs are proposed to make the network input more informative, which contain: joint distance-joint distance motion (JD-JDM), angle-angle motion (A-AM), slow motion coordinates-fast motion coordinates (SMC-FMC). To ensure that features are fully extracted while reducing the number of model parameters, one-dimensional convolutional neural networks (1DCNN) are used to build multi-feature sharing two-stage feature extraction networks. As a result, MSR-Net outperforms state-of-the-art models in accuracy on the JHMDB (86.8%) and SHREC datasets (96.8% on coarse class and 93.8% on fine class). Application experiments were carried out on the HRI platform to demonstrate the effectiveness of MSR-Net in HRI. The video demonstration of the experimental results of the HRI application can be accessed on
Original languageEnglish
Title of host publicationProceedings of International Conference on Advanced Robotics and Mechatronics (ICARM) 2023
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9798350300178
ISBN (Print)9798350300185
Publication statusPublished - 25 Aug 2023
EventInternational Conference on Advanced Robotics and Mechatronics (ICARM) -
Duration: 8 Jul 202310 Jul 2023


ConferenceInternational Conference on Advanced Robotics and Mechatronics (ICARM)

Cite this