RiSSNet: Contrastive Learning Network with a Relaxed Identity Sampling Strategy for Remote Sensing Image Semantic Segmentation

• Published in: Remote sensing (Basel, Switzerland) Vol. 15; no. 13; p. 3427
• Main Authors: Li, Haifeng, Jing, Wenxuan, Wei, Guo, Wu, Kai, Su, Mingming, Liu, Lu, Wu, Hao, Li, Penglong, Qi, Ji
• Format: Journal Article
• Language: English
• Published: Basel MDPI AG 01-07-2023
• Subjects: Analysis; contrastive learning; Data augmentation; Datasets; Hypotheses; Image processing; Image segmentation; Learning; Queues; Remote sensing; remote sensing (RS); Sampling; Satellites; self-supervised learning; Semantic segmentation; Semantics; Similarity; Supervised learning; Teaching methods; Unsupervised learning; China; Germany
• ISSN: 2072-4292; 2072-4292
• DOI: 10.3390/rs15133427

Contrastive learning techniques make it possible to pretrain a general model in a self-supervised paradigm using a large number of unlabeled remote sensing images. The core idea is to pull positive samples defined by data augmentation techniques closer together while pushing apart randomly sampled negative samples to serve as supervised learning signals. This strategy is based on the strict identity hypothesis, i.e., positive samples are strictly defined by each (anchor) sample’s own augmentation transformation. However, this leads to the over-instancing of the features learned by the model and the loss of the ability to fully identify ground objects. Therefore, we proposed a relaxed identity hypothesis governing the feature distribution of different instances within the same class of features. The implementation of the relaxed identity hypothesis requires the sampling and discrimination of the relaxed identical samples. In this study, to realize the sampling of relaxed identical samples under the unsupervised learning paradigm, the remote sensing image was used to show that nearby objects often present a large correlation; neighborhood sampling was carried out around the anchor sample; and the similarity between the sampled samples and the anchor samples was defined as the semantic similarity. To achieve sample discrimination under the relaxed identity hypothesis, the feature loss was calculated and reordered for the samples in the relaxed identical sample queue and the anchor samples, and the feature loss between the anchor samples and the sample queue was defined as the feature similarity. Through the sampling and discrimination of the relaxed identical samples, the leap from instance-level features to class-level features was achieved to a certain extent while enhancing the network’s invariant learning of features. We validated the effectiveness of the proposed method on three datasets, and our method achieved the best experimental results on all three datasets compared to six self-supervised methods.