Emotion classification based on multi physiological signals using hybrid fusion strategy

The demand for emotion recognition is increasing not only in daily life but also in professional domains such as disease diagnosis and clinical rehabilitation. By monitoring non-invasive physiological signals of patients, providing timely feedback on their emotional state can significantly aid in patient recovery and subsequent expert treatment evaluation. In this study, we propose a mutual fusion approach utilizing both feature fusion and decision fusion techniques for emotion classification. Firstly, statistical knowledge-based methods were employed to select physiological signals and extract highly correlated and complementary features for feature-level fusion. Subsequently, a similarity weight, based on feature selection and statistical analysis, was incorporated into the decision fusion process. Our proposed method yielded promising results, achieving 82.5% accuracy and 75.2% macro f1 score for three-class classification (neutral vs. stress vs. amusement), 95.4% accuracy and 94.6% macro f1 score for two-class classification (stress vs. non-stress) using publicly available data from the WESAD dataset. These results demonstrate that employing a mutual fusion approach outperforms single fusion methods in emotion classification tasks.