Automatic recognition of social engagement for children with autism spectrum disorder

Estimating children's engagement levels improves their understanding of their social behaviors, since they can reflect their devotion to social interaction with others. This paper proposes an automatic method to recognize children's engagement levels in a triadic social interaction context. First, an overall metric function containing behavior, cognition, and affective dimensions is proposed to estimate children's multidimensional engagement levels. Then, the automatic feature extraction method based on gaze estimation, facial expression recognition, pose estimation, and object recognition models is illustrated to extract features to compute the engagement levels. Videos of 24 children, including 13 children with autism spectrum disorder (ASD), in triadic social interaction were collected for the engagement recognition experiment and cross-group analysis. The experimental results validate the effectiveness of the proposed automatic feature extraction method compared to human observations. Cross-group analyses revealed significant differences in affective engagement between children with ASD and typical developmental (TD) children.