Deep Multi-Similarity Hashing with semantic-aware preservation for multi-label image retrieval

• Published in: Expert systems with applications Vol. 205; p. 117674
• Main Authors: Qin, Qibing, Xian, Lintao, Xie, Kezhen, Zhang, Wenfeng, Liu, Yu, Dai, Jiangyan, Wang, Chengduan
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01-11-2022
• Subjects: Deep hashing; Higher-order features; Image retrieval; Multi-similarity loss; Semantic-aware similarity; Semantic-aware similarity; Image retrieval; Deep hashing; Multi-similarity loss; Higher-order features
• ISSN: 0957-4174; 1873-6793
• DOI: 10.1016/j.eswa.2022.117674

Recently, many deep supervised hashing has been developed for multi-label image retrieval applications and has already achieved good effects. However, current methods quantify the similarities between image pairs by counting the number of shared labels roughly, totally ignoring the semantic relationships of image pairs. Besides, the sample imbalance of different classes could bias the training process toward majority categories, deteriorating the model performance. To solve the above-mentioned problems, we propose a novel deep supervised hashing framework, named Deep Multi-Similarity Hashing with semantic-aware preserving (DMSH), to generate binary codes with high-level semantic similarity. Specifically, to obtain distinguishing features, Convolutional Neural Networks (CNN) with correlation operators are introduced to capture discriminative semantic features of the input images, including auto-correlation characteristics and cross-correlation characteristics. By jointly exploiting label-level and semantic-level similarity, the semantic-aware similarity module is developed to quantify the high-level semantic similarity of image pairs, distinguishing the differences beyond categories. Furthermore, the multi-similarity loss is incorporated into deep hashing to collect informative pairs efficiently, which could alleviate the impact of imbalanced sample pairs during model training. Extensive comparison experiments on three benchmark datasets demonstrate that our DMSH achieves promising performance with respect to different evaluation metrics.
•CNNs with correlation operators are introduced to capture higher-order features.•A semantic-aware method is proposed to quantify the similarity of image pairs.•Multi-similarity loss is developed to collect informative pairs efficiently.•The state-of-the-art results are reported on three benchmark datasets.