Feature fusion of sEMG and ultrasound signals in hand gesture recognition

Jia Zeng, Yu Zhou, Yicheng Yang, Jiaole Wang, Honghai Liu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Multi-modal sensory fusion is believed to obtain higher accuracy in gesture recognition. Its difficulty lies in mining discriminative features and fusing features from different modalities. Surface electromyography(sEMG) and ultrasound signals are typical signal modalities in gesture recognition. It is expected that the fusion of them can take advantage of the complementarity of electrophysiological information and muscle morphology information. This paper proposed two kinds of feature fusion method. The one is concatenating the manual designed sEMG and ultrasound features, and the other is a convolutional neural network (CNN) based feature exaction and fusion method for sEMG and ultrasound signals. Eight able-bodied subjects were involved to participate in the experiments. In the experiments, four channels of sEMG and A-mode ultrasound signals corresponding to 20 gestures were collected synchronously to evaluate the proposed method. The experimental results demonstrated that the fusion sEMG-ultrasound feature always outperformed the separate sEMG or ultrasound feature regardless of the feature extraction method, and as for fusion sEMG-ultrasound feature, the CNN based method achieve a high accuracy (97.38±1.49%) in 20 gestures, which surpassed the method of concatenating the manual designed features and applying machine learning algorithm (LDA, KNN, SVM).

Original languageEnglish
Title of host publication2020 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2020
PublisherInstitute of Electrical and Electronics Engineers
Pages3911-3916
Number of pages6
ISBN (Electronic)9781728185262
ISBN (Print)9781728185279
DOIs
Publication statusPublished - 14 Dec 2020
Event2020 IEEE International Conference on Systems, Man, and Cybernetics - Toronto, Canada
Duration: 11 Oct 202014 Oct 2020
https://www.ieeesmc.org/conferences/calendar/event/655-2020-ieee-international-conference-on-systems-man-and-cybernetics-smc

Publication series

Name2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC)
PublisherIEEE
ISSN (Print)1062-922X
ISSN (Electronic)2577-1655

Conference

Conference2020 IEEE International Conference on Systems, Man, and Cybernetics
Abbreviated titleSMC 2020
Country/TerritoryCanada
CityToronto
Period11/10/2014/10/20
Internet address

Fingerprint

Dive into the research topics of 'Feature fusion of sEMG and ultrasound signals in hand gesture recognition'. Together they form a unique fingerprint.

Cite this