A hybrid sensor integrating surface electromyography and photoplethysmography for gesture recognition

This study develops a wireless, distributed, and synchronized multimodal sensor system that integrates surface electromyography (sEMG) and photoplethysmography (PPG) signals to address the limitations of unimodal sensing in gesture recognition. The proposed sensor nodes enable synchronous acquisition of sEMG and PPG signals, which are subsequently used for classifying 15 commonly used gestures via machine learning (ML) and deep learning algorithms. Experimental results demonstrate that multimodal fusion significantly enhances recognition performance, improving classification accuracy from 85.3% (sEMG-only) to 95.3% when both modalities are combined. In particular, the recognition accuracy for the “wrist flexion” gesture reaches 99.2%. Additionally, we conducted signal-to-noise ratio (SNR) analysis and experiments under varying muscle contraction intensities, providing preliminary evidence that PPG signals are responsive to muscle activity changes. Correlation analysis and principal component analysis (PCA) further verified the complementary nature of sEMG and PPG signals. To explore the intermodal dependency, we also performed transfer entropy analysis and temporal alignment evaluation. With its low cost and compact size, the proposed system exhibits promising application potential in clinical rehabilitation and human-machine interaction scenarios, such as gesture recognition and muscle activity assessment. Compared to other sEMG-based multimodal systems, the integration of PPG offers a novel perspective toward meeting practical clinical demands.