Facial expression synthesis based on denoising diffusion probabilistic model

Facial expression synthesis has been receiving more attention for research and application in computer vision. The state-of-the-art latent diffusion model (LDM) can generate high-quality images from text prompts. However, to edit the facial expression of existing images, the model can over-editing and remove some identity from the original images. In this study, we build a facial expression synthesis pipeline to edit the original image with different expressions: anger, disgust, contempt, fear, happiness, sadness, surprise, and neutral. Our pipeline includes facial segmentation to extract the necessary area for editing, denoising diffusion probabilistic models (DDPM) with text embedding to generate and control the output expression, and image combining to combine generated image back to the original image. In this paper, we experiment and analyze the potential of DDPM with our method.