Hierarchical knee image synthesis framework for generative adversarial network: data from the Osteoarthritis Initiative

Medical images synthesis is useful to address persistent issues such as the lack of training data diversity and inflexibility of traditional data augmentation faced by medical image analysis researchers when developing their deep learning models. Generative adversarial network (GAN) can generate realistic image to overcome the abovementioned problems. We proposed a GAN model with hierarchical framework (HieGAN) to generate high-quality synthetic knee images as a prerequisite to enable effective training data augmentation for deep learning applications. During the training, the proposed framework embraced attention mechanism before the 256 ×256 scale in generator and discriminator to capture salient information of knee images. Then, a novel pixelwise-spectral normalization configuration was implemented to stabilize the training performance of HieGAN. We evaluated the proposed HieGAN on large scale knee image dataset by using Am Score and Mode Score. The results showed that HieGAN outperformed all relevant state-of-art. Hence, HieGAN can potentially serve as an important milestone to promote future development of more robust deep learning models for knee image segmentation. Future works should extend the image synthesis evaluation to clinical-related Visual Turing Test and synthetic data augmentation for deep learning segmentation task.