Panchromatic and multispectral image fusion for remote sensing and earth observation: concepts, taxonomy, literature review, evaluation methodologies and challenges ahead

Kai Zhang, Feng Zhang, Wenbo Wan, Hui Yu*, Jiande Sun, Javier Del Ser, Eyad Elyan, Amir Hussain

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

46 Downloads (Pure)

Abstract

Panchromatic and multispectral image fusion, termed pan-sharpening, is to merge the spatial and spectral information of the source images into a fused one, which has a higher spatial and spectral resolution and is more reliable for downstream tasks compared with any of the source images. It has been widely applied to image interpretation and pre-processing of various applications. A large number of methods have been proposed to achieve better fusion results by considering the spatial and spectral relationships among panchromatic and multispectral images. In recent years, the fast development of artificial intelligence (AI) and deep learning (DL) has significantly enhanced the development of pan-sharpening techniques. However, this field lacks a comprehensive overview of recent advances boosted by the rise of AI and DL. This paper provides a comprehensive review of a variety of pan-sharpening methods that adopt four different paradigms, i.e., component substitution, multiresolution analysis, degradation model, and deep neural networks. As an important aspect of pan-sharpening, the evaluation of the fused image is also outlined to present various assessment methods in terms of reduced-resolution and full-resolution quality measurement. Then, we conclude this paper by discussing the existing limitations, difficulties, and challenges of pan-sharpening techniques, datasets, and quality assessment. In addition, the survey summarizes the development trends in these areas, which provide useful methodological practices for researchers and professionals. Finally, the developments in pan-sharpening are summarized in the conclusion part. The aim of the survey is to serve as a referential starting point for newcomers and a common point of agreement around the research directions to be followed in this exciting area.

Original languageEnglish
Pages (from-to)227-242
Number of pages16
JournalInformation Fusion
Volume93
Early online date5 Jan 2023
DOIs
Publication statusPublished - 1 May 2023

Keywords

  • image fusion
  • image quality evaluation
  • multispectral image
  • pan-sharpenin
  • panchromatic image
  • UKRI
  • EPSRC
  • EP/M026981/1
  • EP/T021063/1
  • EP/T024917/1

Cite this