A deeper look into remote sensing scene image misclassification by CNNs

As deeper and lighter variations of convolutional neural networks (CNNs) continue to break accuracy and efficiency records, their applications for solving domain-specific challenges continue to widen, particularly in computer vision and pattern recognition. The feat achieved by these end-to-end learning models can be attributed to their ability to extract local and global discriminative features for effective classification. However, in land use and land cover classification (LULC), inner-class variability and outer-class similarity could cause a classifier to confuse one image’s discriminative features with another’s, leading to inefficiency and poor classification. In this work, we deviate from the conventional approach of classifying high-resolution remote sensing images (HRRS) by proposing a framework for comparing and combining images of different simple classes into superclasses based on spatial, textural, and colour similarities. To achieve this, we implement the Bhattacharyya metric for colour-based similarity analysis, a combination of LBPs (Local Binary Pattern), the Earth Mover’s Distance, and Euclidean Distance for the texture and spatial similarity analysis in addition to the structural similarity index (SSIM). A pre-trained CNN model (Xception) is then fine-tuned to classify the superclasses and the original classes of the Aerial Image (AID), the UCM, the Optical Image Analysis and Learning (OPTIMAL-31), and NWPU-RESISC45 datasets. Results show that methodically combining overlapping classes into superclasses reduces the possibility of misclassifications and increases the efficiency of CNNs. The model evaluation further indicates that this approach can boost classifiers’ robustness and significantly reduce the impact of inner-class variability and outer-class similarity on their performance.