A Novel Approach to Visual Search in E-commerce Fashion Using Siamese Neural Network and Multi-Scale CNN

• Published in: 2023 International Electronics Symposium (IES) pp. 460 - 465
• Main Authors: Ma'Rufah, Laila, Karlita, Tita, Sa'Adah, Umi, Fauzi, Willy Achmat
• Format: Conference Proceeding
• Language: English
• Published: IEEE 08-08-2023
• Subjects: e-commerce fashion; Measurement; Multi-Scale CNN; Neural networks; Object detection; Predictive models; Siamese Neural Network; Training; Training data; Visual search; Visualization
• ISSN: 2687-8909
• DOI: 10.1109/IES59143.2023.10242507

The growing number of users on fashion e-retail platforms has highlighted the limitations of the current text-based search system, as users struggle to effectively describe the clothing products they are searching for. To address this challenge, the integration of a visual search system becomes essential. Visual search allows users to find similar products based solely on images. In this research, we propose the utilization of a Siamese Neural Network for implementing visual search in an e-commerce fashion. The Siamese network consists of two identical networks that process different images. By training the network to learn and represent the differences between images as embedding vectors, we enable the generation of similarity scores for pairs of images. To achieve accurate feature extraction, we employ a Multi-Scale CNN approach that captures both high and low-level features. Various experiments are conducted in this study, including the selection of multi-scale CNNs, distance metrics, and diverse data, to determine the network architecture that achieves the highest accuracy. Our findings demonstrate that utilizing the VGG19 model with a shallow layer and fractional distance metrics achieves an impressive accuracy of up to 97%. This indicates the effectiveness of our model in accurately predicting image similarity.