Product image retrieval using category-aware siamese convolutional neural network feature

• Published in: Journal of King Saud University. Computer and information sciences Vol. 34; no. 6; pp. 2680 - 2687
• Main Authors: Rahman, Arif, Winarko, Edi, Mustofa, Khabib
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01-06-2022; Elsevier
• Subjects: Category-aware; Convolutional network; Product retrieval; Category-aware; Product retrieval; Convolutional network
• ISSN: 1319-1578; 2213-1248
• DOI: 10.1016/j.jksuci.2022.03.005

Product image retrieval in the customer-to-shop setting uses similarity learning instead of a predefined distance to address the cross-domain matching problem. Similarity learning can be done using a Siamese convolutional network (SCN) model with pairwise or triplet image sampling. The model training uses product item labels as the target without considering the product category. However, images in the e-shop are inherently have hierarchically structured from the category to the individual image. Therefore, category information should be involved to improve the discriminating factor of the image feature. To accommodate this, we propose a SCN model that involves category and item labels in training to produce the category-aware feature. Our model is based on SCN with modification in training procedure that simultaneously learns the category and item label. Our category-aware Siamese CNN is implemented using MobileNet as the backbone and single-layer network for the mid-feature learner. The results show that our method can improve the accuracy of product image retrieval using SCN based features.